I'll second this... You are free to use Kafka or anything as your data source.
However, your use case does not suggest storm to me. If speed is not essential and exactly once semantics are essential, perhaps you could just use something like a transacted queue to gather client messages and a process reading from such queue, processing and persisting to HBase in a single thread, and removing the message from the queue only after HBase confirms it has persisted the result. Regards, Javier On Jul 1, 2015 6:14 AM, "Enno Shioji" <[email protected]> wrote: > You can use any data source but you have to have *a* data source. In > particular, if you want to have exactly-once semantics you need a data > source that can support that. > > What's the volume of messages you'll be handling though? It may be that > you'll be better off just writing from your server app. instead of going > through Storm. > > > On Wed, Jul 1, 2015 at 8:32 AM, Chandrashekhar Kotekar < > [email protected]> wrote: > >> Hi, >> >> I want to implement Storm for real time message processing in my project. >> I have observed that many people use 'Apache Kafka' along with 'Storm'. >> >> In my project client applications will send messages to server side and >> server side is supposed to authenticate messages, process them and store >> into HBase. Only constraint is that message should not get dropped, each >> and every message must be persisted into HBase, it is fine if it takes few >> minutes to process that message. >> >> I would like to know >> 1) Is it mandatory to use Kafka with Storm? >> 2) What are the advantages of using Kafka with Storm? >> 3) What will happen if I don't use Kafka with storm? >> >> Request you to please enlighten me on usage of Kafka. >> >> Regards, >> Chandrash3khar Kotekar >> Mobile - +91 8600011455 >> > >
