I'm also hitting this issue of too many files open. We too are evaluating embedding ODB into our app and use it as repository to load/unload it on need basis. The application is run as system service(windows/linux service) and due to which we are seeing memory footprint increasing over a period of time.
It would be nice to have some way to bring down the instances completely. Regards, Rohit Regards On Tuesday, 25 August 2015 20:54:28 UTC+5:30, [email protected] wrote: > > Hi, > > I am running a service that creates some snapshot databases (plocal). > I noticed a constant increase in the memory usage and breaks with OOM > after some point. There other issues in the application which I am working > on but also found some related to DB. > > To simulate the behavior, I tried to create 1000 dbs from a 4MB JSON file. > The JSON file is deserialized once to java object and same objects are used > to create all the database instances. > I have a threadpool of size 5 i.e. 5 databases are created in parallel. I > do not see an OOM in the test app but almost froze after 380 instances. > Tests are run using 512 heap size to reproduce the issue early. > > I am using OPartitionedDatabasePool and closing the pool after each > database instance is created and data is written. > However pool.close does not close the storage, also WAL threads and Cache > flush threads are retained for each instance even though those threads are > not working much. > Storages are closed at jvm shutdown for performance reasons and not when > DB instance is not in use. > > The heap usage is growing constantly for each database instance created. > How can I optimize this? > I would assume the application should behave more or less same as at a > time it is working only with 5 instances even though it has already created > 300 instances. > > Also I am getting "max number of files open" in some linux flavors and had > to override the max files open limit. > > *Below is the graphs from visualvm* > > First 5 instances > > > <https://lh3.googleusercontent.com/-PA9IcVmGjy8/VdyC1g7yijI/AAAAAAAAAck/bolFqekEG4c/s1600/at5instances.png> > > 50 instances > > > <https://lh3.googleusercontent.com/-4i2YmajZzpw/VdyDHpyMjrI/AAAAAAAAAcs/DkKTyziPeNs/s1600/at50instances.png> > > 350 Instances > > > <https://lh3.googleusercontent.com/-sVtrYF9u4ak/VdyDKFwmhJI/AAAAAAAAAc0/b07vsBXuIOc/s1600/at350instances.png> > > Threads List at the point of 350 + instances. 2 Threads per instance. > > > <https://lh3.googleusercontent.com/-16gIIu8Gndo/VdyDMLHKIdI/AAAAAAAAAc8/r4rGy_RGH2U/s1600/threadsat350instances.png> > > > <https://lh3.googleusercontent.com/-hyEaDu5jRmo/VdyDOE3c0fI/AAAAAAAAAdE/f-uaDlxstEM/s1600/threadsat350instances1.png> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
