Please have a look at Storm DRPC for this to see if that would help make things easier for you.
On Mar 16, 2017 6:45 AM, "Steve Robert" <srob...@qualys.com> wrote: Hi guys , i have an simple question with an simple scenario is it a good practice to use the bolts for this connects to an external data source ? Here is a concrete example: I would connect to external API to retrieve data for multiple user I have a first Cassandra data source which stores credential users for each user. I created a spout for read database and provide a stream of this accounts To parallelize the downloads of this user's data , I wanted to create a Bolt with parallelism with *fieldsGrouping* strategy for avoid the process of the same message on the same time The Bolt could therefore consume the credential metadata to connecte and get data from the the external API. Do you think this mechanism is acceptable ? I know I could use a first Spout to read credential users and send it to Kafka and create new Spout for read kafka , this spout could then connect to two data sources , the first Kafka for the crendential and the external API to have the users's datas But I find this mechanism cumbersome for a simple scenario Thank for your help ! -- Steve Robert <https://www.linkedin.com/company/qualys> Software Engineer srob...@qualys.com T Qualys, Inc. – Continuous Security Blog <https://qualys.com/blog> | Community <https://community.qualys.com> | Twitter <https://twitter.com/qualys> <https://www.qualys.com/email-banner>