Hi All, I was writing a simple Streaming job to get more understanding about Spark streaming. I am not understanding why the union behaviour in this particular case
*WORKS:* val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words = lines..flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() wordCounts.saveAsTextFiles("all") This works as expected as well as the streams are stored as files *DOESN'T WORK* val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val lines1 = ssc.socketTextStream("localhost", 10000, StorageLevel.MEMORY_AND_DISK_SER) * val words = lines.union(lines1).flatMap(_.split(" "))* val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() wordCounts.saveAsTextFiles("all") In the above case neither the messages are printed nor the files are saved. Am I doing something wrong here? Thanks, Shrikar