# I looked the documents of kafka and I see that there is no way a > consume instance can > read specific messages from partition. >
With Kafka you read messages from the beginning multiple times. Since you say later that you do not have many messages per topic, you can iterate over the message and read the ones that you need. Of course this might not the be the most efficient. > # I have an use case where I need to spawn a topic(single partition) > for each user, > so there would be 10k online users at a time, there would be very > less data per topic. > My questions is, what are limitations of having multiple topics(with > 1 partition), I think > this situation would cause heavy memory consumption and are they any > other limitations?. > Basically the problem boils down to what are the scalability > limitations of having > multiple topics(hardware/software)? > Partitioning the topic helps scale the writes. If you compare kafka to other message brokers, the others might choke when writes happen above a certain rate. By partitioning you are distributing the writes across multiple nodes/broker. You don'nt mention replication but that is relevant as well. It provides redundancy in that , if the primary broker goes down, your messages are still available. regards -- http://khangaonkar.blogspot.com/
