[Spark Streaming] Are SparkListener/StreamingListener callbacks called concurrently?

2018-06-20 Thread Majid Azimi
Hi, What is the concurrency model behind SparkListener or StreamingListener callbacks? 1. Multiple threads might access callbacks simultaneously. 2. Callbacks are guaranteed to be executed by a single thread.(Thread ids might change on consecutive calls, though) I asked the same question on

Do I need to know Scala to take full advantage of spark?

2014-07-30 Thread Majid Azimi
Hi guys I'm very new in spark land comming from old school MapReduce world. I have no idea about scala. Does Java/Python API can compete with native Scala API? Is Spark heavily scala centric and binding for other languages are only for starting up and testing and serious work in spark will