It is not a good idea to store anything on zookeeper other than the metadata. You can use a centralized data-store for storing your logs.
Now for the spout thing , why do you want to remove the files using a spout you can do that using another bolt say "removebolt" which is meant for removing the files from the batch. You can catch the encode-decode exception and do the removal in the catch block. No need to traceback to your spout. Spout is only meant for fetching and supplying data. On Mar 24, 2014 2:35 PM, "M Tarkeshwar Rao" <[email protected]> wrote: > Hi all, > > > > Can you please suggest how FailedException is working in Storm(Trident)? > > Can I send the desired Information to spout using this exception without > disturbing the bolt process(Which raised the exception). > > > > We are processing some files and those files have the data. We are using > file name as tuple and using 10 files batch for processing using Trident. > > We are decoding the data inside file in a bolt and getting some > encode-decode errors. I want to send this error message to the spout to > remove > > those files from batch and replay the remaining batch. > > > > Another strategy we planned to store the failed information as log on > zookeeper and get the log from there . Is this strategy is better or any > risk in it? > > > > Please suggest if you can any other way to do it. > > > > Regards > > Tarkeshwar >
