What is the error you are seeing?
TD
On Thu, May 14, 2015 at 9:00 AM, Erich Ess <er...@simplerelevance.com>
wrote:
> Hi,
>
> Is it possible to setup streams from multiple Kinesis streams and process
> them in a single job? From what I have read, this should be possible,
> however, the Kinesis layer errors out whenever I try to receive from more
> than a single Kinesis Stream.
>
> Here is the code. Currently, I am focused on just getting receivers setup
> and working for the two Kinesis Streams, as such, this code just attempts
> to
> print out the contents of both streams:
>
> implicit val formats = Serialization.formats(NoTypeHints)
>
> val conf = new SparkConf().setMaster("local[*]").setAppName("test")
> val ssc = new StreamingContext(conf, Seconds(1))
>
> val rawStream = KinesisUtils.createStream(ssc, "erich-test",
> "kinesis.us-east-1.amazonaws.com", Duration(1000),
> InitialPositionInStream.TRIM_HORIZON, StorageLevel.MEMORY_ONLY)
> rawStream.map(msg => new String(msg)).print
>
> val loaderStream = KinesisUtils.createStream(
> ssc,
> "dev-loader",
> "kinesis.us-east-1.amazonaws.com",
> Duration(1000),
> InitialPositionInStream.TRIM_HORIZON,
> StorageLevel.MEMORY_ONLY)
>
> val loader = loaderStream.map(msg => new String(msg)).print
>
> ssc.start()
>
> Thanks,
> -Erich
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-Kinesis-Streams-in-a-single-Streaming-job-tp22889.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
