Hi Viktor Am Dienstag, 26. August 2014 11:41:46 UTC+2 schrieb √: > > Hi Christian, > > > On Tue, Aug 26, 2014 at 11:34 AM, Christian Kreutzfeldt <[email protected] > <javascript:>> wrote: > >> Hi Viktor, >> >> thanks for the quick reply. What I am concerned about is the way how to >> read from websocket >> > > Read it from the websocket and store it into Kafka, and have the > Kafka-consumer actor reused for the work-distribution? > Yeah, thought about that as well but ... breaks my architecture somehow. But probably this is the best solution as it enables some other nice features like shutting down the processing actors while the source listener still receives data and inserts it into the kafka topic :-)
> >> .. kafka is easy as I simply pull data and it does not matter when I do >> that, I always receive the next message. But when it comes to websockets, >> the server pushes data and I need to forward the data somehow. When I do >> not consume the message, it is lost. I use the WebSocketClient provided by >> the Jetty framework but that one internally spawns a thread reading from >> the socket forwarding data to a callback provided by me. As far as I >> understood, spawning threads inside actors isn't a great idea :-) ...or am >> I wrong? Do you have any suggestion how to solve this or point me to a >> resource which provides any information about how to handle similar >> problems ... unfortunately google does not help in that case :-( >> > > You could use the Play WebSocket API. > ;-) thanks!! > > >> >> Regarding the kafka consumer, actually I have an implementation which >> does exactly what you described: read single message from kafka, forward >> message, issue next read request via scheduler / directly >> getSelf().tell(..), wait for read request, read single message from kafka >> .... I could keep on reading forever from kafka but that would block the >> actor. But unfortunately, I have to wait for the next read request to >> access kafka. Somehow it is a nice approach especially if a processing >> actor issues that message since it ensures that it has resources available >> for processing the next message. On the other hand it slows down the whole >> process and leaves gaps where nothing is processed. From a software >> architecture aspect this is a clean solution but I need to process data >> quite fast ;-) >> > > You could have the Kafka-actor batch-read up to X messages to not be fully > lock-step with the consumer-actor. > Mmmh, I should try that as well. > > -- > Cheers, > √ > Thanks for your quick replies! Best regards, Christian -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
