The question is how consistent do you need this IP address filter to be? If
you need it to be 100% consistent, then you need to do it the slow way, but
because you are using MongoDB, I assume that you can handle some inconsistency.
The simplest way would be to use the keyed grouping and use a cache that has a
timed expiry that is slightly less than the inconsistency that you can
tolerate. I am not a mongodb expert so I don't know if it can publish edits to
an external pub/sub system, or send edit events back to clients. But that
would just be an efficiency improvement over the cache.
Alternatively it looks like MongoDB does have an async API, so you could do the
lookups asynchronously, if the return order of the events does not matter.
That would not change the overall latency of processing the events nor the load
on MongoDB, but it would increase the throughput of your bolt/topology.
- Bobby
On Thursday, August 6, 2015 3:28 AM, Ritesh Sinha
<[email protected]> wrote:
I need to write a bolt which processes the messages on the basis of IP.
Basically , i have a list of IP's stored in mongoDB.All the messages i get will
contain IP's, So , i need to check if that particular IP is present in the list
or not if it is present then i need to ignore the message else process it.
The list of IP is dynamic .There can be addition or deletion of IPs after the
topology has been deployed to storm.
The problem is either I need to keep the IP is memory(but the changes can't be
reflected once the list is updated) or read it every time from mongoDB, which
makes the processing slow.
Is there any way to implement it efficiently ?
Is it possible to trigger any event if addition or deletion of IP has happened
in mongoDB ?
