Hi, Kafka has partitions and akka can do parallel processing. I have one perfect use-case where I have to read data in parallel. But seems like partitions does not give me any extra info other than partition number and how do i make sure that data_x should always go to x-partition next time and I want to keep it consistent , means if new data_k comes that should create one more partition k, is there is any way of associating some meta data so that next time I should push it back in same and if not present , i can create one more.
may be I can keep created 10,000 partitions and keep associating over and over until my all partitions are consumed. Example : eg: I have twitter handles, how do I make sure to feed all tweets from same handle go to same partition and It should not affect once a new handle gets added. 2. In akka I can keep ready my consumers inside actors and keep listing to partitions in parallel. But for newly added handle how do I create one more actor-child to start listening, I do not want to scan meta data entirely (seems like , after 1mints I can do scan is only way to get this done.) . Keep learning keep moving .....
