Hi I am having a use case where few files in a directory are needed to be processed by a certain bolt x written in Java.
I am setting number of executers and tasks same which is > 1. Say I have 4 executers and tasks. As I understand, these are essentially threads in the worker process. Now I want to make sure that each of the executer / task should process a file uniquely. How to ensure that ? Should I put a synchronised block inside execute method and make sure processing is done in thread safe manner ? This is actually to be done when a topology is launched. As the worker starts, the corresponding bolts will scan a specific directory and process the files previously generated but not processed. In normal scenario, another bolt will pass on file name and path to be processed to this bolt.
