Just to note its an example spout . But I am not sure why it wouldn't allow parallelism of more than 1. For twitter api its http calls thats by increasing the parallelism of the spout you are making more calls to twitter api. I think twitter rate limits based on appid thats the limitation I can see but thats not related to spout parallelism. Even with a single spout instance you can go over your api call rate limit.
-Harsha On Mon, Sep 8, 2014, at 06:35 AM, Vikas Agarwal wrote: That is interesting. However, it won't allow spout parallelism more than 1, right? On Mon, Sep 8, 2014 at 6:56 PM, Harsha <[1][email protected]> wrote: Hi Connie, You can take a look at twittersamplespout in examples [2]https://github.com/apache/incubator-storm/blob/mast er/examples/storm-starter/src/jvm/storm/starter/spout/TwitterSa mpleSpout.java It uses twitter4j to read the api you can make changes to fit your needs. -Harsha On Mon, Sep 8, 2014, at 12:15 AM, Vikas Agarwal wrote: I guess no and it won't make sense to have one because it would limit the parallelism of spout. Twitter stream allows only single connection to the stream. You can use threading to have parallelism in stream consumption but it would be difficult to manage it with spouts. Better solution would be to writer standalone twitter stream listener with multithreading and push messages to Kafka (or some JMS queue) and then consume them using KafkaSpout for instance. It would allow you to increase parallelism of spout by the number of partitions of the topics. On Mon, Sep 8, 2014 at 12:21 PM, Connie Yang <[3][email protected]> wrote: Hi, Is there a spout that stream tweeter feed based a list of hashtags? Thanks, Connie -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters [4]http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 [5]+1 (408) 988-2000 Work [6]+1 (408) 716-2726 Fax -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters [7]http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax References 1. mailto:[email protected] 2. https://github.com/apache/incubator-storm/blob/master/examples/storm-starter/src/jvm/storm/starter/spout/TwitterSampleSpout.java 3. mailto:[email protected] 4. http://www.infoobjects.com/ 5. tel:%2B1%20%28408%29%20988-2000 6. tel:%2B1%20%28408%29%20716-2726 7. http://www.infoobjects.com/
