I tested the same with the below storage close implementation. The result looks good with 1GB heap. With 512 it was running but very slow because of GC overhead.
*After 450 instances* <https://lh3.googleusercontent.com/-Rv0Z4yRucs8/Veazqe5gGrI/AAAAAAAAAdY/FuOJw1E0kWQ/s1600/storageclose.JPG> Collection<OStorage> storages = Orient.instance().getStorages(); for (OStorage storage : storages) { if (!storage.isClosed() && storage.getName().equals(this.instanceId)) { storage.synch(); storage.close(true, false); Orient.instance().unregisterStorageByName(this.instanceId); } } I do not see any logs related to DB recovery when I reload the DB after above steps. Files are also released. I am thinking of using the same for the time being till the Auto Close Defect <https://github.com/orientechnologies/orientdb/issues/3055> is fixed. Is the right way to unload storage? Is there anything else I need to do to ensure proper unload? Regards, Rayees On Monday, August 31, 2015 at 2:42:35 PM UTC+5:30, Rohitdev Kulshrestha wrote: > > I'm also hitting this issue of too many files open. > We too are evaluating embedding ODB into our app and use it as repository > to load/unload it on need basis. > The application is run as system service(windows/linux service) and due to > which we are seeing memory footprint increasing over a period of time. > > It would be nice to have some way to bring down the instances completely. > > Regards, > Rohit > > Regards > > On Tuesday, 25 August 2015 20:54:28 UTC+5:30, [email protected] wrote: >> >> Hi, >> >> I am running a service that creates some snapshot databases (plocal). >> I noticed a constant increase in the memory usage and breaks with OOM >> after some point. There other issues in the application which I am working >> on but also found some related to DB. >> >> To simulate the behavior, I tried to create 1000 dbs from a 4MB JSON >> file. The JSON file is deserialized once to java object and same objects >> are used to create all the database instances. >> I have a threadpool of size 5 i.e. 5 databases are created in parallel. I >> do not see an OOM in the test app but almost froze after 380 instances. >> Tests are run using 512 heap size to reproduce the issue early. >> >> I am using OPartitionedDatabasePool and closing the pool after each >> database instance is created and data is written. >> However pool.close does not close the storage, also WAL threads and Cache >> flush threads are retained for each instance even though those threads are >> not working much. >> Storages are closed at jvm shutdown for performance reasons and not when >> DB instance is not in use. >> >> The heap usage is growing constantly for each database instance created. >> How can I optimize this? >> I would assume the application should behave more or less same as at a >> time it is working only with 5 instances even though it has already created >> 300 instances. >> >> Also I am getting "max number of files open" in some linux flavors and >> had to override the max files open limit. >> >> *Below is the graphs from visualvm* >> >> First 5 instances >> >> >> <https://lh3.googleusercontent.com/-PA9IcVmGjy8/VdyC1g7yijI/AAAAAAAAAck/bolFqekEG4c/s1600/at5instances.png> >> >> 50 instances >> >> >> <https://lh3.googleusercontent.com/-4i2YmajZzpw/VdyDHpyMjrI/AAAAAAAAAcs/DkKTyziPeNs/s1600/at50instances.png> >> >> 350 Instances >> >> >> <https://lh3.googleusercontent.com/-sVtrYF9u4ak/VdyDKFwmhJI/AAAAAAAAAc0/b07vsBXuIOc/s1600/at350instances.png> >> >> Threads List at the point of 350 + instances. 2 Threads per instance. >> >> >> <https://lh3.googleusercontent.com/-16gIIu8Gndo/VdyDMLHKIdI/AAAAAAAAAc8/r4rGy_RGH2U/s1600/threadsat350instances.png> >> >> >> <https://lh3.googleusercontent.com/-hyEaDu5jRmo/VdyDOE3c0fI/AAAAAAAAAdE/f-uaDlxstEM/s1600/threadsat350instances1.png> >> >> -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
