Timouts problem can be same in both storm and trident? Topologies can be as big as you want them to, but I have personally > preferred small ones because of the same problem that you are facing. > Timeouts. > For you, I'd recommend that after your trident state inserts to DB, send > an ack and don't do any further emits. > Let the lower part of your diagram be a separate topology that queries > what Trident wrote and processes it. > In Trident state, I am maintaining some data in
memory as well - basically the aggregated data of around 100 tuples emitted by previous bolt, to know whether the processing is done or not. This information (processed tuples) will be used for further processing > Also, each REST call section and the "each" bolt before it could be moved > into the spout. > If I move those to spout, won't I be missing all the parallelism required to call the REST API's Please do let me know.
