As per the documentation : http://spark.apache.org/docs/latest/streaming-programming-guide.html#input-dstreams-and-receivers
"if you want to receive multiple streams of data in parallel in your streaming application, you can create multiple input DStreams (discussed further in the Performance Tuning section). This will create multiple receivers which will simultaneously receive multiple data streams. But note that a Spark worker/executor is a long-running task, hence it occupies one of the cores allocated to the Spark Streaming application. Therefore, it is important to remember that a Spark Streaming application needs to be allocated enough cores (or threads, if running locally) to process the received data, as well as to run the receiver(s)." "it is important to remember that a Spark Streaming application needs to be allocated enough cores" In this reference of "enough cores", what will be the minimum cores for a receiver in Spark Streaming ? Can we say 2 cores per Receiver ? Kindly correct me to understand it. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-minimum-cores-for-a-Receiver-tp25307.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
