Well, you could that (Stage information) is an ASCII representation of the
WebUI (running on port 4040). Since you set local[4] you will have 4
threads for your computation, and since you are having 2 receivers, you are
left with 2 threads to process ((0 + 2) <-- This 2 is your 2 threads.) And
the other /2 means you are having 2 tasks in that stage (with id 0).
Thanks
Best Regards
On Tue, Jun 23, 2015 at 1:21 PM, bit1...@163.com <bit1...@163.com> wrote:
> Hi,
>
> I have a spark streaming application that runs locally with two receivers,
> some code snippet is as follows:
>
> conf.setMaster("local[4]")
>
> //RPC Log Streaming
> val rpcStream = KafkaUtils.createStream[String, String, StringDecoder,
> StringDecoder](ssc, consumerParams, topicRPC, StorageLevel.MEMORY_ONLY)
> RPCLogStreamProcessor.process(rpcStream, taskConfBroadcast)
>
> //HTTP Log Streaming
> val httpStream = KafkaUtils.createStream[String, String, StringDecoder,
> StringDecoder](ssc, consumerParams, topicHTTP, StorageLevel.MEMORY_ONLY)
> HttpLogStreamProcessor.process(httpStream, taskConfBroadcast)
>
>
> There is a log information showing on the console in red color
> [Stage 0:> (0 + 2) / 2]
> It appears, then disappear, and then appear, disappear...
>
> For the above code, if I only have rpc streaming and comment the
> httpStream, then it disappear. I don't know how it occurs and how to
> suppress it
>
> ------------------------------
> bit1...@163.com
>
