Hi Artem, Thanks for response.
We did try it in both ways (tx and notx), and we faced that issue that too many objects are loaded in memory (both scenarios). I will execute the same operation with notx and will provide new charts... среда, 25 июня 2014 г., 12:10:56 UTC+3 пользователь Artem Orobets написал: > > Hi Andrey, > > I took a look at your charts and it looks like you have too much records > loaded in memory. > I suppose the problem is that transaction becomes too big. So try to use > OrientGraphNoTx for massive insert. If you still need to do an insertion in > transaction you could divide it on several transactions by doing a commit() > sometimes. > > Best regards, > Artem Orobets > > * Orient Technologiesthe Company behind OrientDB* > > > 2014-06-25 9:56 GMT+03:00 Андрей Логинов <[email protected] > <javascript:>>: > >> Hello, Artem, >> >> >> >> *>> Could you be more specific? What class instance do you mean? And >> where it is kept? It would be great if you could point me at code, probably >> we could avoid such behavior.* >> Seems like I made a mistake by claiming this. I haven't found anything >> suspicious in code. However I still face the memory issue.. I spent some >> time profiling the application and here are my results with allocated 2GB >> memory for heap. There is definitely something that cannot by collected by >> GC. >> Please take a look at the memory usage and GC activity: >> http://snag.gy/kfFLH.jpg >> And here is heap gistogram: http://snag.gy/rkRUb.jpg >> >> Please take a look at LinkedHashMap$entry it is increasing all the time. >> It is not used in our code so I assume this is something from orient-core. >> Do you have any ideas what it could be? >> >> >> >> *>> Have you tried to check if value is set correctly with >> isValidataionEnabled call? * >> Yes I tries, it returns 'false' after disabling it. So seems working >> fine. However I couldn't notice that it helped a lot... So another question >> is: is validation performed when I'm using blueprints API? If so, does it >> help when I disable it using .getRawGraph().set... method and then continue >> using blueprint API to add vertexes and edges? >> >> One more note, after disabling validation, I started getting next >> messages in console (somewhere about 10-15 messages during the whole >> process): >> 2014 9:53:42 AM com.orientechnologies.common.log.OLogManager log >> WARNING: Update of collection pointer is received but collection is not >> registered >> >> >> Maybe there is anything else I could disable to perform massive insertion >> without allocating additional GBs heap memory? >> >> >> вторник, 24 июня 2014 г., 8:52:43 UTC+3 пользователь Artem Orobets >> написал: >>> >>> Hi Andrey, >>> >>> We found, that after adding a new vertex, a new class instance is >>>> created each time to validate the vertex. However it is not realized after >>>> all. >>> >>> Could you be more specific? What class instance do you mean? And where >>> it is kept? It would be great if you could point me at code, probably we >>> could avoid such behavior. >>> >>> 2) I'm afraid Blueprints API doesn't aware of schema, it is more orient >>> specific, so that API doesn't allow to configure it. >>> >>> 3) The setValidationEnabled method change the state of db where it was >>> invoked. So *<OrientBaseGraph>.getRawGraph(* >>> *).setValidationEnabled(false);* should work. Have you tried to check >>> if value is set correctly with isValidataionEnabled call? >>> >>> Best regards, >>> Artem Orobets >>> >>> * Orient Technologiesthe Company behind OrientDB* >>> >>> >>> 2014-06-23 18:30 GMT+03:00 Андрей Логинов <[email protected]>: >>> >>>> Good day, >>>> >>>> We've faced several issue while trying massive insertions (900k >>>> vertexes, millions of edges): >>>> It is becoming possible only after adding additional memory to client's >>>> VM (up to 4GB) . We found, that after adding a new vertex, a new class >>>> instance is created each time to validate the vertex. However it is not >>>> realized after all. >>>> >>>> At the same time I've found next article in wiki: https://github.com/ >>>> orientechnologies/orientdb/wiki/Performance-Tuning-Graph. I noticed >>>> that there is special method: >>>> >>>> graph.setValidationEnabled(false); >>>> >>>> The thing is - I'm using blueprint API and can access this method only via >>>> <OrientBaseGraph>.getRawGraph().setValidationEnabled(false); >>>> >>>> >>>> However it seems like that I should work with returned object further in >>>> order to make this change work. >>>> >>>> >>>> So my questions are: >>>> 1) how to make orientDB client (via blueprint API) work without adding >>>> additional memory? >>>> 2) how to disable validation using this API? >>>> 3) how to convert result of setValidationEnabled back to OrientBaseGraph? >>>> >>>> >>>> >>>> >>>> Thanks in advance. >>>> >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OrientDB" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
