The XGBoost integration with Spark is currently only supported for RDDs, there is a ticket for dataframe and folks calm to be working on it.
On Aug 14, 2016 8:15 PM, "Jacek Laskowski" <ja...@japila.pl> wrote: > Hi, > > I've never worked with the library and speaking about sbt setup only. > > It appears that the project didn't release 2.11-compatible jars (only > 2.10) [1] so you need to build the project yourself and uber-jar it > (using sbt-assembly plugin). > > [1] https://spark-packages.org/package/rotationsymmetry/sparkxgboost > > Pozdrawiam, > Jacek Laskowski > ---- > https://medium.com/@jaceklaskowski/ > Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > > On Sun, Aug 14, 2016 at 7:13 AM, janardhan shetty > <janardhan...@gmail.com> wrote: > > Any leads how to do acheive this? > > > > On Aug 12, 2016 6:33 PM, "janardhan shetty" <janardhan...@gmail.com> > wrote: > >> > >> I tried using sparkxgboost package in build.sbt file but it failed. > >> Spark 2.0 > >> Scala 2.11.8 > >> > >> Error: > >> [warn] > >> http://dl.bintray.com/spark-packages/maven/ > rotationsymmetry/sparkxgboost/0.2.1-s_2.10/sparkxgboost-0.2. > 1-s_2.10-javadoc.jar > >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: > >> [warn] :: FAILED DOWNLOADS :: > >> [warn] :: ^ see resolution messages for details ^ :: > >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: > >> [warn] :: > >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(src) > >> [warn] :: > >> rotationsymmetry#sparkxgboost;0.2.1-s_2.10!sparkxgboost.jar(doc) > >> > >> build.sbt: > >> > >> scalaVersion := "2.11.8" > >> > >> libraryDependencies ++= { > >> val sparkVersion = "2.0.0-preview" > >> Seq( > >> "org.apache.spark" %% "spark-core" % sparkVersion % "provided", > >> "org.apache.spark" %% "spark-sql" % sparkVersion % "provided", > >> "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided", > >> "org.apache.spark" %% "spark-mllib" % sparkVersion % "provided" > >> ) > >> } > >> > >> resolvers += "Spark Packages Repo" at > >> "http://dl.bintray.com/spark-packages/maven"; > >> > >> libraryDependencies += "rotationsymmetry" % "sparkxgboost" % > >> "0.2.1-s_2.10" > >> > >> assemblyMergeStrategy in assembly := { > >> case PathList("META-INF", "MANIFEST.MF") => > >> MergeStrategy.discard > >> case PathList("javax", "servlet", xs @ _*) => > >> MergeStrategy.first > >> case PathList(ps @ _*) if ps.last endsWith ".html" => > >> MergeStrategy.first > >> case "application.conf" => > >> MergeStrategy.concat > >> case "unwanted.txt" => > >> MergeStrategy.discard > >> > >> case x => val oldStrategy = (assemblyMergeStrategy in assembly).value > >> oldStrategy(x) > >> > >> } > >> > >> > >> > >> > >> On Fri, Aug 12, 2016 at 3:35 PM, janardhan shetty < > janardhan...@gmail.com> > >> wrote: > >>> > >>> Is there a dataframe version of XGBoost in spark-ml ?. > >>> Has anyone used sparkxgboost package ? > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
