architecture though experiment: what is the advantage of using kafka with spark streaming?

I noticed that many people are using Kafka and spark streaming. Can some one provide a couple of use case I image some possible use cases might be Is the purpose using Kafka 1. provide some buffering? 2. implementing some sort of load balancing for the over all system? 3. Provide filtering

Re: architecture though experiment: what is the advantage of using kafka with spark streaming?

Kafka provides buffering, ordering, decoupling of producers from multiple consumers. So pretty much any time you have requirements for asynchronous process, fault tolerance, and/or a common view of the order of events across multiple consumers kafka is worth a look. Spark provides a much richer