Hi all, *My requirement is *To process and index a huge data set to Elasticsearch.
For each tuple that spout emits, about 100 child tuples gets emitted. Each of them needs to be processed and indexed to Elasticsearch. Once the children gets indexed, parent tuple that spout emitted also need to be indexed by querying the children index which is already indexed. I am able to achieve the whole functionality using normal storm. But since the processing of the tuple is too long, I had to disable guaranteed message processing. Since processing tuples repeatedly is a costly operation for me, I decided to use storm trident as it claims to be support exactly once processing. The problem here is I could not achieve the complete functionality with trident. I have to index children and based on the bulk response that I get from elasticsearch, I need to emit some more tuples for further processing. I understand that we can use trident state for doing batch insert to elasticsearch. But based on the response I could not emit from trident state. Please help me solve this - "Batch insert and emit based on response using Trident." Thanks Pratyusha
