Btw. I think you'll find a report about this from a few years ago in the Collections JIRA. Just search for my name.
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Renè Glanzer <[email protected]> > To: Commons Users List <[email protected]> > Sent: Monday, June 15, 2009 12:00:54 PM > Subject: Re: [collections] LRUMap Problem ConcurrentModificationException > > Yes of course, > > the code with the time based cache systems was set up with an ordniary > HashMap. But when rapidly seach requests are > generated the time based mechanism fails to delete old entries - cause > they are not old enough - and so the cache will > raise up until the entire VM memory is used. Then i found the LRUMap > switched to it and bang Exception in my delete method. > > When i switch back to HashMap the code runs well - as currently on the > productive system - except the open memory leak. > > René > > 2009/6/15 Leon Rosenberg : > > just out of curiosity have you tried the same code with a standard hashmap? > > > > Leon > > > > On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzerwrote: > >> Hello, > >> > >> side note accepted :-) > >> > >> In my class I checked the get, put and remove methods. All are > >> synchronized. > >> As you can see also the code which wants to delete the elements is > synchronized. > >> So I've no clue where the "ConcurrentModificationException" is comming > >> from > :-( > >> > >> Thanks René > >> > >> 2009/6/15 Leon Rosenberg : > >>> Hello, > >>> > >>> on a side note, generics make reading of code easier :-) > >>> > >>> you haven't posted the whole code, but have you (double)checked that > >>> all other acesses to store are synchronized? > >>> > >>> regards > >>> Leon > >>> > >>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzerwrote: > >>>> I'm calling the remove() method on the iterator. I know when i call > >>>> the remove on the map itself it will cause the problem with my > >>>> currently running iterator, but i'm aware of that. > >>>> > >>>> Here is the code block which should delete old entries: > >>>> > >>>> store: is the LRUMap > >>>> birth: is a long which keeps the creation time when this is set to 0 > >>>> the item should be deleted > >>>> > >>>> public void removeAllExpiredItems() > >>>> { > >>>> synchronized(this.store) > >>>> { > >>>> Iterator it=this.store.keySet().iterator(); > >>>> while(it.hasNext()) > >>>> { > >>>> Object key=it.next(); > >>>> Object o=this.get(key); > >>>> if(o != null) > >>>> { > >>>> Item iEntry=(Item)this.store.get(key); > >>>> if(iEntry.birth==0) > >>>> it.remove(); //Here the exception occurs > >>>> } > >>>> } > >>>> this.setLastCleanDate(new Date()); //only to know when the > >>>> deleter run the last time > >>>> } > >>>> } > >>>> > >>>> Thanks for any help :-) > >>>> René > >>>> > >>>> 2009/6/15 James Carman : > >>>>> Are you calling remove() on the iterator or on the map itself? > >>>>> > >>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer > wrote: > >>>>>> Hello, > >>>>>> > >>>>>> is there still no help for me? > >>>>>> Is somebody able to explain me, why i get this > >>>>>> "java.util.ConcurrentModificationException" > >>>>>> on iterating and calling remove() on the LRUMap? > >>>>>> > >>>>>> Please > >>>>>> René > >>>>>> > >>>>>> 2009/6/10 Renè Glanzer : > >>>>>>> Hello Ted, > >>>>>>> > >>>>>>> thanks for the fast response. I understand that simultaneously puts > >>>>>>> and gets do not cause the exception cause they are synchronized in my > >>>>>>> class. > >>>>>>> And my stated code-fragment is the part which wants to delete the > >>>>>>> entries from the underlying LRUMap. And my code is wrapped in the > >>>>>>> synchronized block. But i think the LRUMap herselfe also iterates the > >>>>>>> entries and this maybe happens at the same time when my iterator is > >>>>>>> checking and trying to delete elements. My class is not implementing > >>>>>>> the LRUMap but has one LRUMap as a variable. > >>>>>>> So your solution to override the removeLRU(Entry) methode would not > >>>>>>> help me....unfortunatelly :-( > >>>>>>> > >>>>>>> For now i really do not know how to solve the problem in an easy way > >>>>>>> without refractoring the entire code? > >>>>>>> Any other solutions? > >>>>>>> Thanks René > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> 2009/6/9 Ted Dunning : > >>>>>>>> I apologize for not reading your thread with great care, but I think > that > >>>>>>>> your problem is pretty clear. > >>>>>>>> > >>>>>>>> The issue is not to do with gets and puts overlapping. The issue is > that a > >>>>>>>> put or remove happened during the life of your iterator. > >>>>>>>> > >>>>>>>> One way to avoid this is to synchronize the entire method that does > >>>>>>>> the > >>>>>>>> deletion of old elements. To speed that up, you might just > >>>>>>>> synchronize > the > >>>>>>>> scan for elements to delete and collect them into a list. Then you > >>>>>>>> can > >>>>>>>> delete them outside the loop. Since your scan is probably pretty > >>>>>>>> fast, > this > >>>>>>>> may be sufficient. With very high levels of updates and threading, > >>>>>>>> it > would > >>>>>>>> cause problems. > >>>>>>>> > >>>>>>>> Another option is to clone the table. I believe that some > implementations > >>>>>>>> of LRUMap in Lucene do copy-on-write semantics, but I don't think > >>>>>>>> that > >>>>>>>> collections has these. Without that, cloning will be as slow or > >>>>>>>> slower > than > >>>>>>>> your scan so the first option would be better. > >>>>>>>> > >>>>>>>> I am curious, however, why you didn't make use of the built-in > capabilities > >>>>>>>> of the LRUMap to help you with this. Notably, you should probably > >>>>>>>> just > >>>>>>>> over-ride the removeLRU(Entry) method and set the scanUntilRemovable > flag. > >>>>>>>> I think that this would take the place of your loop and would be > >>>>>>>> much > more > >>>>>>>> efficient. > >>>>>>>> > >>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer > wrote: > >>>>>>>> > >>>>>>>>> ... Additionally to the maximum number of the LRUMap i also > implemented a > >>>>>>>>> thread which periodicly checks the age of the entries and deletes > >>>>>>>>> them > >>>>>>>>> when they are too old. > >>>>>>>>> > >>>>>>>>> For this i generated an Iterator which delivers me each entry and if > >>>>>>>>> the creation date is too old i call iterator.remove(). > >>>>>>>>> But exactly on this line i get an > >>>>>>>>> "java.util.ConcurrentModificationException" although i wrapped all > >>>>>>>>> access to the map with synchronized blocks. So every put and get > >>>>>>>>> methods are synchronized. > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: [email protected] > >>>>>> For additional commands, e-mail: [email protected] > >>>>>> > >>>>>> > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
