Hello Cheung, Happy New Year!
No, I did not configure Hive on my machine. Even I have tried not setting the HADOOP_HOME but getting the same error. Regards, _________________________________ *Md. Rezaul Karim* BSc, MSc PhD Researcher, INSIGHT Centre for Data Analytics National University of Ireland, Galway IDA Business Park, Dangan, Galway, Ireland Web: http://www.reza-analytics.eu/index.html <http://139.59.184.114/index.html> On 29 December 2016 at 19:16, Felix Cheung <felixcheun...@hotmail.com> wrote: > Any reason you are setting HADOOP_HOME? > > From the error it seems you are running into issue with Hive config likely > with trying to load hive-site.xml. Could you try not setting HADOOP_HOME > > > ------------------------------ > *From:* Md. Rezaul Karim <rezaul.ka...@insight-centre.org> > *Sent:* Thursday, December 29, 2016 10:24:57 AM > *To:* spark users > *Subject:* Issue with SparkR setup on RStudio > > > Dear Spark users, > > I am trying to setup SparkR on RStudio to perform some basic data > manipulations and ML modeling. However, I am a strange error while > creating SparkR session or DataFrame that says: > java.lang.IllegalArgumentException > Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState. > > According to Spark documentation at http://spark.apache.org/docs/ > latest/sparkr.html#starting-up-sparksession, I don’t need to configure > Hive path or related variables. > > I have the following source code: > > SPARK_HOME = "C:/spark-2.1.0-bin-hadoop2.7" > HADOOP_HOME= "C:/spark-2.1.0-bin-hadoop2.7/bin/" > > library(SparkR, lib.loc = c(file.path(Sys.getenv("SPARK_HOME"), "R", > "lib"))) > sparkR.session(appName = "SparkR-DataFrame-example", master = "local[*]", > sparkConfig = list(spark.sql.warehouse.dir="E:/Exp/", spark.driver.memory > = "8g"), enableHiveSupport = TRUE) > > # Create a simple local data.frame > localDF <- data.frame(name=c("John", "Smith", "Sarah"), age=c(19, 23, 18)) > # Convert local data frame to a SparkDataFrame > df <- createDataFrame(localDF) > print(df) > head(df) > sparkR.session.stop() > > Please note that the HADOOP_HOME contains the ‘*winutils.exe’* file. The > details of the eror is as follows: > > Error in handleErrors(returnStatus, conn) : > java.lang.IllegalArgumentException: Error while instantiating > 'org.apache.spark.sql.hive.HiveSessionState': > > at > org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981) > > at > org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110) > > at > org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109) > > at > org.apache.spark.sql.api.r.SQLUtils$$anonfun$setSparkContextSessionConf$2.apply(SQLUtils.scala:67) > > at > org.apache.spark.sql.api.r.SQLUtils$$anonfun$setSparkContextSessionConf$2.apply(SQLUtils.scala:66) > > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) > > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > > at > scala.collection.AbstractIterator.foreach(Iterator.scala:1336) > > at > scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > > at scala.collection.Traversabl > > > Any kind of help would be appreciated. > > > > > Regards, > _________________________________ > *Md. Rezaul Karim* BSc, MSc > PhD Researcher, INSIGHT Centre for Data Analytics > National University of Ireland, Galway > IDA Business Park, Dangan, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> >
