Making spark read from sources other than HDFS
Hi team, I'm running spark in cluster mode. We have a custom file storage in our organisation. Can I plug in data from these custom sources (Non HDFS like...) Can you please shed some light in this aspect, like where do I start, should I have to tweak the spark source code (Where exactly do I look out for?) Thank you. -- With Thanks and Regards, Ramprakash Ramamoorthy, Chennai, India
Re: Illegal access error when initializing SparkConf
Team, Got this fixed. After so much of juggling, when I replaced JRE7 with JRE8, things started working the way intended. Cheers! On Wed, Jul 1, 2015 at 7:07 PM, Ramprakash Ramamoorthy youngestachie...@gmail.com wrote: Team, I'm just playing around with spark and mllib. Installed scala and spark, versions mentioned below. Scala - 2.11.7 Spark - 1.4.0 (Did an mvn package with -Dscala-2.11) I'm trying to run the Java classification, clustering examples that came along with the documentation. However, I'm getting the illegal access error when I'm trying to initialize the SparkConf object. Please find the error trace below : *Exception in thread main java.lang.IllegalAccessError: tried to access method scala.collection.mutable.HashSet.init()V from class org.apache.spark.util.Utils$* * at org.apache.spark.util.Utils$.init(Utils.scala:195)* * at org.apache.spark.util.Utils$.clinit(Utils.scala)* * at org.apache.spark.SparkConf.init(SparkConf.scala:58)* * at MultinomialLogisticRegressionExample.main(MultinomialLogisticRegressionExample.java:15)* How do I go about this? Did some googling and couldn't conclude. Please help and thank you in advance. -- With Thanks and Regards, Ramprakash Ramamoorthy, India. -- With Thanks and Regards, Ramprakash Ramamoorthy, India.
Illegal access error when initializing SparkConf
Team, I'm just playing around with spark and mllib. Installed scala and spark, versions mentioned below. Scala - 2.11.7 Spark - 1.4.0 (Did an mvn package with -Dscala-2.11) I'm trying to run the Java classification, clustering examples that came along with the documentation. However, I'm getting the illegal access error when I'm trying to initialize the SparkConf object. Please find the error trace below : *Exception in thread main java.lang.IllegalAccessError: tried to access method scala.collection.mutable.HashSet.init()V from class org.apache.spark.util.Utils$* * at org.apache.spark.util.Utils$.init(Utils.scala:195)* * at org.apache.spark.util.Utils$.clinit(Utils.scala)* * at org.apache.spark.SparkConf.init(SparkConf.scala:58)* * at MultinomialLogisticRegressionExample.main(MultinomialLogisticRegressionExample.java:15)* How do I go about this? Did some googling and couldn't conclude. Please help and thank you in advance. -- With Thanks and Regards, Ramprakash Ramamoorthy, India.
