After checking the heap dump it seems that the refReverseMap map in the SoftRefFileCache doesn't get cleaned. The LocalFileName objects hold by this map were occupying more then 80% of the memory.
2013/11/13 Béla Hullár <[email protected]> > Hi All, > > We've been developing a system where we move around a lot of files. We > started to use vfs, because sometimes these files are on a remote > filesystem and vfs is really a great library which enables us to handle > these cases uniform. > > As the system was started to be used very heavily recently we experienced > memory issues, OutOfMemoryErrors. After investigating the problem it seems > the cause of the problem was the FilesCaches of the manager which was > growing without limits. We have few questions regarding this: > > 1, is this infinite growing of a SoftRefFilesCache a bug or a feature? > javadoc says if the jvm needs memory then the cache will free. But this one > by one removal in the maintenance thread of the cache doesn't seem to be > very efficient. > > 2, we started to use LRUFilesCache, because with that we can limit the > number of the cached entries. Indeed it seems it has much lower memory > usage (according the set limit), but checking a heap dump showed that our > LRUCache contained around ~21k entries however its size was set to 500. Do > you have some idea how it's possible? > > I created a small test program which can reproduce the issue: > > https://svn.code.sf.net/p/screeningbee/code/trunk/vfs-cache-test/src/main/java/ch/systemsx/bee/vfscachetest/CacheSizeChecker.java > > if you set the testroot for a directory contains a lot of files and limit > the heapsize (eg. -Xmx128m) then it will fail with the default > SoftRefCache and with WeekRefCache and it works with LRU an NullFilesCache > Of course in the real application we have lot more memory, but it was also > eaten up after a while. > > we use java 7 and vfs 2.0 > > Any help would be appreciated! > Bela > > > -- Üdvözlettel: Hullár Béla
