Hello Sumit -

I could see that SparkConf() specification is not being mentioned in your
program. But rest looks good.



Output:


By the way, I have used the README.md template
https://gist.github.com/jxson/1784669

Thanks & Regards,
Gokula Krishnan* (Gokul)*

On Tue, Sep 20, 2016 at 2:15 AM, Chawla,Sumit <sumitkcha...@gmail.com>
wrote:

> Hi All
>
> I am trying to test a simple Spark APP using scala.
>
>
> import org.apache.spark.SparkContext
>
> object SparkDemo {
>   def main(args: Array[String]) {
>     val logFile = "README.md" // Should be some file on your system
>
>     // to run in local mode
>     val sc = new SparkContext("local", "Simple App", 
> ""PATH_OF_DIRECTORY_WHERE_COMPILED_SPARK_PROJECT_FROM_GIT")
>
>     val logData = sc.textFile(logFile).cache()
>     val numAs = logData.filter(line => line.contains("a")).count()
>     val numBs = logData.filter(line => line.contains("b")).count()
>
>
>     println("Lines with a: %s, Lines with b: %s".format(numAs, numBs))
>
>   }
> }
>
>
> When running this demo in IntelliJ, i am getting following error:
>
>
> java.lang.IllegalStateException: unread block data
>       at 
> java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(ObjectInputStream.java:2449)
>       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1385)
>       at 
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
>       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
>       at 
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
>       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
>       at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
>       at 
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
>       at 
> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
>       at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:253)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>       at java.lang.Thread.run(Thread.java:745)
>
>
> I guess its associated with task not being deserializable.  Any help will be 
> appreciated.
>
>
>
> Regards
> Sumit Chawla
>
>

Reply via email to