Are you using the Commons libs in your app? then you need to depend on them directly, and not rely on them accidentally being provided by Spark. There is no trial and error; you must declare all the dependencies you use in your own code.
Otherwise perhaps you are not actually running with the Spark assembly JAR at runtime but somehow only trying to run against the Spark jar itself. This has none of the dependencies that Spark needs. On Tue, Dec 9, 2014 at 3:46 AM, sivarani <[email protected]> wrote: > Dear All, > > I am using spark streaming, It was working fine when i was using spark1.0.2, > now i repeatedly getting few issue > > Like class not found, i am using the same pom.xml with the updated version > for all spark modules > i am using spark-core,streaming, streaming with kafka modules.. > > Its constantly keeps throwing errors for no commons-configuation, > commons-langs, logging > > How to get all the dependencies for running spark streaming.. Is there any > way or we just have to find by trial and error methord? > > my pom dependencies > > <dependencies> > <dependency> > <groupId>javax.servlet</groupId> > <artifactId>servlet-api</artifactId> > <version>2.5</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core_2.10</artifactId> > <version>1.0.2</version> > </dependency> > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming_2.10</artifactId> > <version>1.0.2</version> > </dependency> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming-kafka_2.10</artifactId> > <version>1.0.2</version> > </dependency> > > <dependency> > <groupId>org.slf4j</groupId> > <artifactId>slf4j-log4j12</artifactId> > <version>1.7.5</version> > </dependency> > <dependency> > <groupId>commons-logging</groupId> > <artifactId>commons-logging</artifactId> > <version>1.1.1</version> > </dependency> > <dependency> > <groupId>commons-configuration</groupId> > <artifactId>commons-configuration</artifactId> > <version>1.6</version> > </dependency> > > Am i missing something here? > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-1-1-Maven-dependency-tp20590.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
