On Tuesday 25 August 2009 22:53:54 Mike Bush wrote: > I havn't managed to get full GC's but I think it's possible that there > is still a memory leak, today is the first time in ages i've been able > to fetch the root of freenetindex.
Ok. I get OOMs searching for "encrypted message" in wanna ... shouldn't the memory usage be proportional to the number of valid matches, not to the vast amount of XML in a given gigantic index file? Originally I tried this search because phrase searches seemed to be giving false positives, showing pages that appeared not to contain the phrase in question. > > "message" worked fine but "encrypted" threw a SplitfileException, > probably at about 46%. There was a problem with the error throwing > which caused the search not to notice exceptions thrown whilst > fetching, that would cause what you saw if you got the exception too. > I fixed that yesterday so I'm hoping you are running a version from > before that. What causes a SplitFileException? You mean a FetchException with a splitfile error code - too many errors in blocks or fatal errors in blocks. Caused by blocks failing. :) > > SSK at > US6gHsNApDvyShI~sBHGEOplJ3pwZUDhLqTAas6rO4c,3jeU5OwV0-K4B6HRBznDYGvpu2PRUuwL0V110rn-~8g,AQACAAE/freenet-index-2/index_50.xml > could not be opened: FetchException:Not enough data > found:null:-1:null:false:null:Splitfile error > > I have also got a ConcurrentModificationException a couple of times > searching for the same thing on both indexes and I'm looking into it. > > java.util.ConcurrentModificationException > -- java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) > -- java.util.HashMap$KeyIterator.next(HashMap.java:828) > -- plugins.Library.ui.MainPage.cleanUpPages(MainPage.java:42) > -- plugins.Library.ui.MainPage.processPostRequest(MainPage.java:123) > -- > plugins.Library.ui.MainPageToadlet.handleMethodPOST(MainPageToadlet.java:89) > -- sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > -- > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > -- > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > -- java.lang.reflect.Method.invoke(Method.java:597) > -- freenet.clients.http.ToadletContextImpl.handle(ToadletContextImpl.java:428) > -- > freenet.clients.http.SimpleToadletServer$SocketHandler.run(SimpleToadletServer.java:744) > -- freenet.support.PooledExecutor$MyThread.run(PooledExecutor.java:228) > > > > 2009/8/25 Matthew Toseland <toad at amphibian.dyndns.org>: > > On Saturday 15 August 2009 16:54:12 Matthew Toseland wrote: > >> [16:03:57] <toad_> mikeb_: also i'm suspicious of the phrase search > >> [16:04:13] <toad_> "test subject" gives http://127.0.0.1:8888/SSK at > >> aYWBb6zo2AM13XCNhsmmRKMANEx6PG~C15CWjdZziKA,X1pAG4EIqR1gAiyGFVZ1iiw-uTlh460~rFACJ7ZHQXk,AQACAAE/fms-106/ > >> which doesn't seem to include that prhase > >> [16:04:37] * toad_ searches for "encrypted message" which definitely > >> exists on that page > >> [16:05:47] <mikeb_> toad_ : i'll have to deal with this later i need to > >> pop out > >> > >> "encrypted message" searched for with both indexes stalls: > >> > >> "encrypted message"@bookmark:freenetindex bookmark:wanna : Busy > >> "encrypted message"@bookmark:freenetindex : Busy > >> ? ? ? encrypted ? ? ? Fetching Subindex > >> ? ? ? 46% > >> ? ? ? message ? ? ? ? Done > >> ? ? ? "encrypted message"@bookmark:wanna ? ? ?Done > >> > > Searching for "encrypted message", with no quotes i.e. a union search, on > > freenetindex only, with a 256MB memory limit, sticks at 46% on "encrypted", > > apparently forever. CPU usage is less than 100%, and "message" is shown as > > Done. > > > > Searching for "encrypted message" (no quotes again) on both indexes with a > > 192MB limit gives constant Full GC's. Thread dumps show no trace of Library. > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20090826/cc40c9e5/attachment.pgp>
