I'm not clear at what point you are modifying this set? You mentioned that you have a text file of food to exclude ... Do you use this to populate the set which you then subsequently modify? If so what is the logic (in simple terms) which dictates whether elements are checked, added or removed from the set?
It sounds like there could be many different ways to tackle your problem (distributed locks, consistent hashing, rdbm transactions etc), but I think,we would all need to know a bit more about your setup On Thursday, 20 November 2014, Kushan Maskey < [email protected]> wrote: > > I have a scenario, > > I have a common project where I have a synchronized function to validate > whether a text file contains a string. Example, that text file contains say > a list of food that needs to be excluded. I have a data coming through > kafka and storm which contain list of food. > > The function i created is a synchronized SET with all these food. When i > get some kind of food in my data I look up to see if it needs to be > excluded form getting inserted into the database. Everything worked in my > local environment but when i deploy this code in a clustered environment, > exclusion is a hit or miss. Now the data that gets loaded is not correct > coz the food that is supposed to be excluded still exists. I am 100% sure > it is because of the thread safe issue of that function. How do i achieve > this functionality in the clustered environment. Please advice. Thanks. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates <http://mmillerassociates.com/> > [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');> >
