You need to keep track of state in your bolt before writing. Yes, it is indeed quite a chore. For "exactly once", particularly when coordinating with external systems such as databases or output queues, Storm is, ah, not exactly the best fit. Unless keeping track of processed events to avoid duplication is feasible in your use case.
Regards, Javier On Fri, Jul 3, 2015 at 6:44 AM, jinhong lu <[email protected]> wrote: > Hi, > > I build a topo to get messages from by kafka, and then grep some > keyword, if fit, write to local file. > > I use OpaqueTridentKafkaSpout of storm-kafka to ensure the tuple will not > miss or repeate, but consider one situation: when writing message to local > file, and some error occurse(for example, not enough space). At this > moment, some messages have writen to local file, and others not, if the > spout resend the message, the message will be write twice. > > how to do that?thanks. > > jinhong_lu > -- Javier González Nicolini
