Thanks Vincent for the input. Not sure I understand your suggestion. Please clarify.
Few words about my use case : When the user watches a video, I get the position events from the user that indicates how much they have completed viewing and at a certain point, I mark that Video as complete and persist that info to cassandra. How do I notify the user that it was marked complete? Are you suggesting I write the completed events to kafka(different topic) and the akka consumer could read from this? There could be many completed events from different users in this topic. So the akka consumer should pretty much do what a spark streaming does to process this without the knowledge of the kafka offset. So not sure what you mean by kafka offsets will do the job, how will the akka consumer know the kafka offset? On Mon, Nov 28, 2016 at 12:52 PM, vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > You don't need actors to do kafka=>spark processing=>kafka > Why do you need to notify the akka producer ? If you need to get back the > processed message in your producer, then implement an akka consummer in > your akka app and kafka offsets will do the job > > 2016-11-28 21:46 GMT+01:00 shyla deshpande <deshpandesh...@gmail.com>: > >> Thanks Daniel for the response. >> >> I am planning to use Spark streaming to do Event Processing. I will have >> akka actors sending messages to kafka. I process them using Spark streaming >> and as a result a new events will be generated. How do I notify the akka >> actor(Message producer) that a new event has been generated? >> >> >> >> On Mon, Nov 28, 2016 at 9:51 AM, Daniel van der Ende < >> daniel.vandere...@gmail.com> wrote: >> >>> Well, I would say it depends on what you're trying to achieve. Right now >>> I don't know why you are considering using Akka. Could you please explain >>> your use case a bit? >>> >>> In general, there is no single correct answer to your current question >>> as it's quite broad. >>> >>> Daniel >>> >>> On Mon, Nov 28, 2016 at 9:11 AM, shyla deshpande < >>> deshpandesh...@gmail.com> wrote: >>> >>>> My data pipeline is Kafka --> Spark Streaming --> Cassandra. >>>> >>>> Can someone please explain me when would I need to wrap akka around the >>>> spark streaming app. My knowledge of akka and the actor system is poor. >>>> Please help! >>>> >>>> Thanks >>>> >>> >>> >>> >>> -- >>> Daniel >>> >> >> >
