Hello everyone, I'm building a topology in which I'm counting connected devices. Each device can send a start and a stop message, both with a same unique ID (to match the device).
Due to the high throughput of messages, I'm storing a state (in my case, a list of connected devices) in RAM, shared across all workers, to check and count devices. I'm also storing each start message in HBase, using Put in bulk (and also removing when I got end messages). I don't want to count using HBase (too slow). I'd like to be able, in case of a worker / task failure, to restore the state in RAM by reading the values that are into HBase. Problem is, I have no idea on how to implement this with Trident. I found this PDF: http://didata.us/assets.d/DiData-StormRedis-Portfolio.pdf, that seems to be a good way of doing it, but only with Storm. I need to achieve exactly-once processing thanks to Trident. My questions is, can I achieve all this using Trident ? Is the idea of combining RAM + HBase is wrong ? Many thanks, Quentin
