Re: SPARK-1.6 build with HIVE
Spark 1.6 works fine with Hive 2. I did not know there was a restriction
there.
I assume you are talking about Spark using Hive metastore?
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.6.1
/_/
Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_25)
Type in expressions to have them evaluated.
Type :help for more information.
Spark context available as sc.
SQL context available as sqlContext.
scala> sql("show databases").collect.foreach(println)
[accounts]
[asehadoop]
[default]
[iqhadoop]
[mytable_db]
[oraclehadoop]
[test]
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*
http://talebzadehmich.wordpress.com
On 1 April 2016 at 04:47, guoqing0...@yahoo.com.hk.INVALID <
guoqing0...@yahoo.com.hk.invalid> wrote:
> Hi, I'd like to know is the SPARK-1.6 only support the hive-0.13 or can
> build with higher versions like 1.x ?
>
> --
> guoqing0...@yahoo.com.hk
>
Re: Spark 1.6 Build
thx for mentioning the build requirement But actually it is -*D*scala-2.11 (i.e. -D for java property instead of -P for profile) details: We can see this in the pom.xml scala-2.11 scala-2.11 2.11.7 2.11 So the scala-2.11 profile is activated by detecting the scala-2.11 system property being set 2015-11-24 10:01 GMT-08:00 Ted Yu : > See also: > > https://repository.apache.org/content/repositories/orgapachespark-1162/org/apache/spark/spark-core_2.11/v1.6.0-preview2/ > > w.r.t. building locally, please specify -Pscala-2.11 > > Cheers > > On Tue, Nov 24, 2015 at 9:58 AM, Stephen Boesch wrote: > >> HI Madabhattula >> Scala 2.11 requires building from source. Prebuilt binaries are >> available only for scala 2.10 >> >> From the src folder: >> >>dev/change-scala-version.sh 2.11 >> >> Then build as you would normally either from mvn or sbt >> >> The above info *is* included in the spark docs but a little hard to find. >> >> >> >> 2015-11-24 9:50 GMT-08:00 Madabhattula Rajesh Kumar >> : >> >>> Hi Ted, >>> >>> I'm not able find "spark-core_2.11 and spark-sql_2.11 jar files" in >>> above link. >>> >>> Regards, >>> Rajesh >>> >>> On Tue, Nov 24, 2015 at 11:03 PM, Ted Yu wrote: >>> See: http://search-hadoop.com/m/q3RTtF1Zmw12wTWX/spark+1.6+preview&subj=+ANNOUNCE+Spark+1+6+0+Release+Preview On Tue, Nov 24, 2015 at 9:31 AM, Madabhattula Rajesh Kumar < mrajaf...@gmail.com> wrote: > Hi Prem, > > Thank you for the details. I'm not able to build. I'm facing some > issues. > > Any repository link, where I can download (preview version of) 1.6 > version of spark-core_2.11 and spark-sql_2.11 jar files. > > Regards, > Rajesh > > On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure > wrote: > >> you can refer..: >> https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn >> >> >> On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < >> mrajaf...@gmail.com> wrote: >> >>> Hi, >>> >>> I'm not able to build Spark 1.6 from source. Could you please share >>> the steps to build Spark 1.16 >>> >>> Regards, >>> Rajesh >>> >> >> > >>> >> >
Re: Spark 1.6 Build
See also: https://repository.apache.org/content/repositories/orgapachespark-1162/org/apache/spark/spark-core_2.11/v1.6.0-preview2/ w.r.t. building locally, please specify -Pscala-2.11 Cheers On Tue, Nov 24, 2015 at 9:58 AM, Stephen Boesch wrote: > HI Madabhattula > Scala 2.11 requires building from source. Prebuilt binaries are > available only for scala 2.10 > > From the src folder: > >dev/change-scala-version.sh 2.11 > > Then build as you would normally either from mvn or sbt > > The above info *is* included in the spark docs but a little hard to find. > > > > 2015-11-24 9:50 GMT-08:00 Madabhattula Rajesh Kumar : > >> Hi Ted, >> >> I'm not able find "spark-core_2.11 and spark-sql_2.11 jar files" in above >> link. >> >> Regards, >> Rajesh >> >> On Tue, Nov 24, 2015 at 11:03 PM, Ted Yu wrote: >> >>> See: >>> >>> http://search-hadoop.com/m/q3RTtF1Zmw12wTWX/spark+1.6+preview&subj=+ANNOUNCE+Spark+1+6+0+Release+Preview >>> >>> On Tue, Nov 24, 2015 at 9:31 AM, Madabhattula Rajesh Kumar < >>> mrajaf...@gmail.com> wrote: >>> Hi Prem, Thank you for the details. I'm not able to build. I'm facing some issues. Any repository link, where I can download (preview version of) 1.6 version of spark-core_2.11 and spark-sql_2.11 jar files. Regards, Rajesh On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure wrote: > you can refer..: > https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn > > > On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < > mrajaf...@gmail.com> wrote: > >> Hi, >> >> I'm not able to build Spark 1.6 from source. Could you please share >> the steps to build Spark 1.16 >> >> Regards, >> Rajesh >> > > >>> >> >
Re: Spark 1.6 Build
HI Madabhattula Scala 2.11 requires building from source. Prebuilt binaries are available only for scala 2.10 >From the src folder: dev/change-scala-version.sh 2.11 Then build as you would normally either from mvn or sbt The above info *is* included in the spark docs but a little hard to find. 2015-11-24 9:50 GMT-08:00 Madabhattula Rajesh Kumar : > Hi Ted, > > I'm not able find "spark-core_2.11 and spark-sql_2.11 jar files" in above > link. > > Regards, > Rajesh > > On Tue, Nov 24, 2015 at 11:03 PM, Ted Yu wrote: > >> See: >> >> http://search-hadoop.com/m/q3RTtF1Zmw12wTWX/spark+1.6+preview&subj=+ANNOUNCE+Spark+1+6+0+Release+Preview >> >> On Tue, Nov 24, 2015 at 9:31 AM, Madabhattula Rajesh Kumar < >> mrajaf...@gmail.com> wrote: >> >>> Hi Prem, >>> >>> Thank you for the details. I'm not able to build. I'm facing some >>> issues. >>> >>> Any repository link, where I can download (preview version of) 1.6 >>> version of spark-core_2.11 and spark-sql_2.11 jar files. >>> >>> Regards, >>> Rajesh >>> >>> On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure >>> wrote: >>> you can refer..: https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < mrajaf...@gmail.com> wrote: > Hi, > > I'm not able to build Spark 1.6 from source. Could you please share > the steps to build Spark 1.16 > > Regards, > Rajesh > >>> >> >
Re: Spark 1.6 Build
Hi Ted, I'm not able find "spark-core_2.11 and spark-sql_2.11 jar files" in above link. Regards, Rajesh On Tue, Nov 24, 2015 at 11:03 PM, Ted Yu wrote: > See: > > http://search-hadoop.com/m/q3RTtF1Zmw12wTWX/spark+1.6+preview&subj=+ANNOUNCE+Spark+1+6+0+Release+Preview > > On Tue, Nov 24, 2015 at 9:31 AM, Madabhattula Rajesh Kumar < > mrajaf...@gmail.com> wrote: > >> Hi Prem, >> >> Thank you for the details. I'm not able to build. I'm facing some issues. >> >> Any repository link, where I can download (preview version of) 1.6 >> version of spark-core_2.11 and spark-sql_2.11 jar files. >> >> Regards, >> Rajesh >> >> On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure wrote: >> >>> you can refer..: >>> https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn >>> >>> >>> On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < >>> mrajaf...@gmail.com> wrote: >>> Hi, I'm not able to build Spark 1.6 from source. Could you please share the steps to build Spark 1.16 Regards, Rajesh >>> >>> >> >
Re: Spark 1.6 Build
See: http://search-hadoop.com/m/q3RTtF1Zmw12wTWX/spark+1.6+preview&subj=+ANNOUNCE+Spark+1+6+0+Release+Preview On Tue, Nov 24, 2015 at 9:31 AM, Madabhattula Rajesh Kumar < mrajaf...@gmail.com> wrote: > Hi Prem, > > Thank you for the details. I'm not able to build. I'm facing some issues. > > Any repository link, where I can download (preview version of) 1.6 > version of spark-core_2.11 and spark-sql_2.11 jar files. > > Regards, > Rajesh > > On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure wrote: > >> you can refer..: >> https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn >> >> >> On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < >> mrajaf...@gmail.com> wrote: >> >>> Hi, >>> >>> I'm not able to build Spark 1.6 from source. Could you please share the >>> steps to build Spark 1.16 >>> >>> Regards, >>> Rajesh >>> >> >> >
Re: Spark 1.6 Build
Hi Prem, Thank you for the details. I'm not able to build. I'm facing some issues. Any repository link, where I can download (preview version of) 1.6 version of spark-core_2.11 and spark-sql_2.11 jar files. Regards, Rajesh On Tue, Nov 24, 2015 at 6:03 PM, Prem Sure wrote: > you can refer..: > https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn > > > On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < > mrajaf...@gmail.com> wrote: > >> Hi, >> >> I'm not able to build Spark 1.6 from source. Could you please share the >> steps to build Spark 1.16 >> >> Regards, >> Rajesh >> > >
Re: Spark 1.6 Build
you can refer..: https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/building-spark.html#building-with-buildmvn On Tue, Nov 24, 2015 at 7:16 AM, Madabhattula Rajesh Kumar < mrajaf...@gmail.com> wrote: > Hi, > > I'm not able to build Spark 1.6 from source. Could you please share the > steps to build Spark 1.16 > > Regards, > Rajesh >
