We have an application that uses reactive-kafka to consume messages from a kafka topic, do some validations and transformations on the messages, and then persist to two different datastores. We are using two different reactive kafka consumer streams for each of the datastores, using different consumer-group-ids. Each of the stream can be enabled/disabled via configuration in the application.
When we run either one of the consumer streams, they seem to run fine and are able to consume messages and keep up with the topic reasonably well. But when both streams are run together, we see a pretty high latency (4-5 minutes) and both consumers are unable to keep up with the messages from the topic. We are publishing about 150000 messages/minute to the topic. We are making sure that both consumers are managing blocking where applicable, and are using separate dedicated dispatchers for the same. Any ideas what could be going on ? -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
