Hi Andrey, I took a look at your charts and it looks like you have too much records loaded in memory. I suppose the problem is that transaction becomes too big. So try to use OrientGraphNoTx for massive insert. If you still need to do an insertion in transaction you could divide it on several transactions by doing a commit() sometimes.
Best regards, Artem Orobets * Orient Technologiesthe Company behind OrientDB* 2014-06-25 9:56 GMT+03:00 Андрей Логинов <[email protected]>: > Hello, Artem, > > > > *>> Could you be more specific? What class instance do you mean? And where > it is kept? It would be great if you could point me at code, probably we > could avoid such behavior.* > Seems like I made a mistake by claiming this. I haven't found anything > suspicious in code. However I still face the memory issue.. I spent some > time profiling the application and here are my results with allocated 2GB > memory for heap. There is definitely something that cannot by collected by > GC. > Please take a look at the memory usage and GC activity: > http://snag.gy/kfFLH.jpg > And here is heap gistogram: http://snag.gy/rkRUb.jpg > > Please take a look at LinkedHashMap$entry it is increasing all the time. > It is not used in our code so I assume this is something from orient-core. > Do you have any ideas what it could be? > > > > *>> Have you tried to check if value is set correctly with > isValidataionEnabled call? * > Yes I tries, it returns 'false' after disabling it. So seems working fine. > However I couldn't notice that it helped a lot... So another question is: > is validation performed when I'm using blueprints API? If so, does it help > when I disable it using .getRawGraph().set... method and then continue > using blueprint API to add vertexes and edges? > > One more note, after disabling validation, I started getting next messages > in console (somewhere about 10-15 messages during the whole process): > 2014 9:53:42 AM com.orientechnologies.common.log.OLogManager log > WARNING: Update of collection pointer is received but collection is not > registered > > > Maybe there is anything else I could disable to perform massive insertion > without allocating additional GBs heap memory? > > > вторник, 24 июня 2014 г., 8:52:43 UTC+3 пользователь Artem Orobets написал: >> >> Hi Andrey, >> >> We found, that after adding a new vertex, a new class instance is >>> created each time to validate the vertex. However it is not realized after >>> all. >> >> Could you be more specific? What class instance do you mean? And where it >> is kept? It would be great if you could point me at code, probably we could >> avoid such behavior. >> >> 2) I'm afraid Blueprints API doesn't aware of schema, it is more orient >> specific, so that API doesn't allow to configure it. >> >> 3) The setValidationEnabled method change the state of db where it was >> invoked. So *<OrientBaseGraph>.getRawGraph(* >> *).setValidationEnabled(false);* should work. Have you tried to check if >> value is set correctly with isValidataionEnabled call? >> >> Best regards, >> Artem Orobets >> >> * Orient Technologiesthe Company behind OrientDB* >> >> >> 2014-06-23 18:30 GMT+03:00 Андрей Логинов <[email protected]>: >> >>> Good day, >>> >>> We've faced several issue while trying massive insertions (900k >>> vertexes, millions of edges): >>> It is becoming possible only after adding additional memory to client's >>> VM (up to 4GB) . We found, that after adding a new vertex, a new class >>> instance is created each time to validate the vertex. However it is not >>> realized after all. >>> >>> At the same time I've found next article in wiki: https://github.com/ >>> orientechnologies/orientdb/wiki/Performance-Tuning-Graph. I noticed >>> that there is special method: >>> >>> graph.setValidationEnabled(false); >>> >>> The thing is - I'm using blueprint API and can access this method only via >>> <OrientBaseGraph>.getRawGraph().setValidationEnabled(false); >>> >>> However it seems like that I should work with returned object further in >>> order to make this change work. >>> >>> >>> So my questions are: >>> 1) how to make orientDB client (via blueprint API) work without adding >>> additional memory? >>> 2) how to disable validation using this API? >>> 3) how to convert result of setValidationEnabled back to OrientBaseGraph? >>> >>> >>> >>> Thanks in advance. >>> >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
