Re: Large number of very small streams

2016-09-22 Thread Ravi Sharma
Hi ivan, I assume you are trying to do per user stream so that you can process each user's event in same sequence as they arrive. Is this correct assumption? if yes then with in storm you can manage this even if you read from one kafka topic using one spout and output events on one stream. just

Re: Large number of very small streams

2016-09-21 Thread Ambud Sharma
Two solutions: 1. You can group users by some sort of classification and create topics based on that then for each user the consumer can check if it's interested in the topic and consumer or reject the messages. 2. If each user writes a lot of data then you can use the concept of key based

Large number of very small streams

2016-09-21 Thread Ivan Gozali
Hi everyone, I'm very new to Storm, and have read various documentation but haven't started using it. I have a use case where I could potentially have many users producing data points that are accumulated in one huge, single Kafka topic/Kinesis stream, and I was going to use Storm to "route"