spark1.4.0, streaming + kafka, throw org.apache.spark.util.TaskCompletionListenerException

2015-07-19 Thread Wwh
hi I test spark streaming and kafka,the applicaion like this:import kafka.serializer.StringDecoder import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} import org.apache.spark.streaming.kafka.KafkaUtils import net.sf.json.JSONObject /** * Created by asu

RE: Spark1.4 application throw java.lang.NoClassDefFoundError: javax/servlet/FilterRegistration

2015-07-19 Thread Wwh
it is caused by conflict of dependency,spark-core1.4 need dependency of javax.servlet-api:3.0.1it can be resolved to add dependency to built.sbt as that: "javax.servlet" % "javax.servlet-api" % "3.0.1", From: wwyando...@hotmail.com To: user@spark.apache.org Subject: Spark1.4 application throw j

Spark1.4 application throw java.lang.NoClassDefFoundError: javax/servlet/FilterRegistration

2015-07-18 Thread Wwh
hi I have build a spark application with IDEA. when run SparkPI , IDEA throw exception as that : Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/FilterRegistration at org.spark-project.jetty.servlet.ServletContextHandler.(ServletContextHandler.java:136) at org.sp

Running spark1.4 inside intellij idea HttpServletResponse - ClassNotFoundException

2015-06-15 Thread Wwh
name := "SparkLeaning" version := "1.0" scalaVersion := "2.10.4" //scalaVersion := "2.11.2" libraryDependencies ++= Seq( //"org.apache.hive"% "hive-jdbc" % "0.13.0" //"io.spray" % "spray-can" % "1.3.1", //"io.spray" % "spray-routing" % "1.3.1", "io.spray" % "spray-testkit" % "1.3.1" % "t