Hi All, I have set-up storm cluster with one nimbus server and three supervisors. Published three topologies which does different calculations as follows
- Topology1 : Reads raw data from MongoDB, do some calculations and finally store back the result. - Topology2 : Reads the result of topology1 and do some calculations and publish results to a queue - Topology3 : Consumes output of topology2 from the queue, calls a REST Service, get reply from REST service, update result in MongoDB collection, finally send an email. I relatively new to storm, hence looking for an expert advice on the following questions 1. Currently configuration to connect MongoDB, MySql, Mq, REST urls are hard-coded in java file. The file is published to cluster along with the topology. I found it is not good practice to customize source files for each customer. Is there a way to externalize all configurations, for example a config.json, that can be referred by all elements of topology? 2. Wanted to extract logs at each stage [Spouts and Bolts], how do I configure log4j.xml and place is it in the runtime of cluster? 3. Is it right to execute blocking call like REST call from a bolt? Any help would be much appreciated. -- Thanks, Nageswara Rao.V *"The LORD reigns"*
