RandomGenerator class not found exception
My sbt file for the project includes this: libraryDependencies ++= Seq( org.apache.spark %% spark-core % 1.1.0, org.apache.spark %% spark-mllib % 1.1.0, org.apache.commons % commons-math3 % 3.3 ) = Still I am getting this error: java.lang.NoClassDefFoundError: org/apache/commons/math3/random/RandomGenerator = The jar at location: ~/.m2/repository/org/apache/commons/commons-math3/3.3 contains the random generator class: $ jar tvf commons-math3-3.3.jar | grep RandomGenerator org/apache/commons/math3/random/RandomGenerator.class org/apache/commons/math3/random/UniformRandomGenerator.class org/apache/commons/math3/random/SynchronizedRandomGenerator.class org/apache/commons/math3/random/AbstractRandomGenerator.class org/apache/commons/math3/random/RandomGeneratorFactory$1.class org/apache/commons/math3/random/RandomGeneratorFactory.class org/apache/commons/math3/random/StableRandomGenerator.class org/apache/commons/math3/random/NormalizedRandomGenerator.class org/apache/commons/math3/random/JDKRandomGenerator.class org/apache/commons/math3/random/GaussianRandomGenerator.class Please help
Re: RandomGenerator class not found exception
Add this jar http://mvnrepository.com/artifact/org.apache.commons/commons-math3/3.3 while creating the sparkContext. sc.addJar(/path/to/commons-math3-3.3.jar) And make sure it is shipped and available in the environment tab (4040) Thanks Best Regards On Mon, Nov 17, 2014 at 1:54 PM, Ritesh Kumar Singh riteshoneinamill...@gmail.com wrote: My sbt file for the project includes this: libraryDependencies ++= Seq( org.apache.spark %% spark-core % 1.1.0, org.apache.spark %% spark-mllib % 1.1.0, org.apache.commons % commons-math3 % 3.3 ) = Still I am getting this error: java.lang.NoClassDefFoundError: org/apache/commons/math3/random/RandomGenerator = The jar at location: ~/.m2/repository/org/apache/commons/commons-math3/3.3 contains the random generator class: $ jar tvf commons-math3-3.3.jar | grep RandomGenerator org/apache/commons/math3/random/RandomGenerator.class org/apache/commons/math3/random/UniformRandomGenerator.class org/apache/commons/math3/random/SynchronizedRandomGenerator.class org/apache/commons/math3/random/AbstractRandomGenerator.class org/apache/commons/math3/random/RandomGeneratorFactory$1.class org/apache/commons/math3/random/RandomGeneratorFactory.class org/apache/commons/math3/random/StableRandomGenerator.class org/apache/commons/math3/random/NormalizedRandomGenerator.class org/apache/commons/math3/random/JDKRandomGenerator.class org/apache/commons/math3/random/GaussianRandomGenerator.class Please help
Re: RandomGenerator class not found exception
Include the commons-math3/3.3 in class path while submitting jar to spark cluster. Like.. spark-submit --driver-class-path maths3.3jar --class MainClass --master spark cluster url appjar On Mon, Nov 17, 2014 at 1:55 PM, Ritesh Kumar Singh [via Apache Spark User List] ml-node+s1001560n19055...@n3.nabble.com wrote: My sbt file for the project includes this: libraryDependencies ++= Seq( org.apache.spark %% spark-core % 1.1.0, org.apache.spark %% spark-mllib % 1.1.0, org.apache.commons % commons-math3 % 3.3 ) = Still I am getting this error: java.lang.NoClassDefFoundError: org/apache/commons/math3/random/RandomGenerator = The jar at location: ~/.m2/repository/org/apache/commons/commons-math3/3.3 contains the random generator class: $ jar tvf commons-math3-3.3.jar | grep RandomGenerator org/apache/commons/math3/random/RandomGenerator.class org/apache/commons/math3/random/UniformRandomGenerator.class org/apache/commons/math3/random/SynchronizedRandomGenerator.class org/apache/commons/math3/random/AbstractRandomGenerator.class org/apache/commons/math3/random/RandomGeneratorFactory$1.class org/apache/commons/math3/random/RandomGeneratorFactory.class org/apache/commons/math3/random/StableRandomGenerator.class org/apache/commons/math3/random/NormalizedRandomGenerator.class org/apache/commons/math3/random/JDKRandomGenerator.class org/apache/commons/math3/random/GaussianRandomGenerator.class Please help -- If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055.html To start a new topic under Apache Spark User List, email ml-node+s1001560n1...@n3.nabble.com To unsubscribe from Apache Spark User List, click here http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=1code=bGVhcm5pbmdzLmNoaXR0dXJpQGdtYWlsLmNvbXwxfC03NzExMjUwMg== . NAML http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055p19057.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: RandomGenerator class not found exception
As you are using sbt ..u need not put in ~/.m2/repositories for maven. Include the jar explicitly using the option --driver-class-path while submitting the jar to spark cluster On Mon, Nov 17, 2014 at 7:41 PM, Ritesh Kumar Singh [via Apache Spark User List] ml-node+s1001560n1907...@n3.nabble.com wrote: It's still not working. Keep getting the same error. I even deleted the commons-math3/* folder containing the jar. And then under the directory org/apache/commons/ made a folder called 'math3' and put the commons-math3-3.3.jar in it. Still its not working. I also tried sc.addJar(/path/to/jar) within spark-shell and in my project sourcefile It still didn't import the jar at both locations. More Any fixes? Please help On Mon, Nov 17, 2014 at 2:14 PM, Chitturi Padma [hidden email] http://user/SendEmail.jtp?type=nodenode=19073i=0 wrote: Include the commons-math3/3.3 in class path while submitting jar to spark cluster. Like.. spark-submit --driver-class-path maths3.3jar --class MainClass --master spark cluster url appjar On Mon, Nov 17, 2014 at 1:55 PM, Ritesh Kumar Singh [via Apache Spark User List] [hidden email] http://user/SendEmail.jtp?type=nodenode=19057i=0 wrote: My sbt file for the project includes this: libraryDependencies ++= Seq( org.apache.spark %% spark-core % 1.1.0, org.apache.spark %% spark-mllib % 1.1.0, org.apache.commons % commons-math3 % 3.3 ) = Still I am getting this error: java.lang.NoClassDefFoundError: org/apache/commons/math3/random/RandomGenerator = The jar at location: ~/.m2/repository/org/apache/commons/commons-math3/3.3 contains the random generator class: $ jar tvf commons-math3-3.3.jar | grep RandomGenerator org/apache/commons/math3/random/RandomGenerator.class org/apache/commons/math3/random/UniformRandomGenerator.class org/apache/commons/math3/random/SynchronizedRandomGenerator.class org/apache/commons/math3/random/AbstractRandomGenerator.class org/apache/commons/math3/random/RandomGeneratorFactory$1.class org/apache/commons/math3/random/RandomGeneratorFactory.class org/apache/commons/math3/random/StableRandomGenerator.class org/apache/commons/math3/random/NormalizedRandomGenerator.class org/apache/commons/math3/random/JDKRandomGenerator.class org/apache/commons/math3/random/GaussianRandomGenerator.class Please help -- If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055.html To start a new topic under Apache Spark User List, email [hidden email] http://user/SendEmail.jtp?type=nodenode=19057i=1 To unsubscribe from Apache Spark User List, click here. NAML http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: Re: RandomGenerator class not found exception http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055p19057.html Sent from the Apache Spark User List mailing list archive http://apache-spark-user-list.1001560.n3.nabble.com/ at Nabble.com. -- If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055p19073.html To start a new topic under Apache Spark User List, email ml-node+s1001560n1...@n3.nabble.com To unsubscribe from Apache Spark User List, click here http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=1code=bGVhcm5pbmdzLmNoaXR0dXJpQGdtYWlsLmNvbXwxfC03NzExMjUwMg== . NAML http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/RandomGenerator-class-not-found-exception-tp19055p19086.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
