Hi, I'm afraid you have at least one memory problem in your code: the temp HashMap will grow a lot and will probably take you to an OOM, that could be a good explanation for the problem you are seeing.
You can try to make it a bit lighter storing in the map only RIDs instead of full vertices, and then reload them when needed. I hope it helps Thanks Luigi 2016-06-13 12:13 GMT+02:00 K Major <[email protected]>: > Okay, > Here is the complete loading code : https://gitlab.com/snippets/20918 > > > - Each node has name, age, age and pagerank properties are from a > class Person. > - Each edge has a weight property. > - 2 types of edges are created : hates and likes. > > > I'm first loading all the vertices and then all the edges. > > I load the data from a hash map of the following form. > {5:[6,7], 7:[8]... ]: means that the vertex 5 is connected to vertices 6 > and 7, vertex 7 is connected to vertex 8, etc. > > > Le lundi 13 juin 2016 09:38:56 UTC+2, Luigi Dell'Aquila a écrit : >> >> Hi, >> >> It's strange, with low memory you can expect some performance degradation >> (eg. a little slowdown), but not a system hang. >> Could you please post the code you use for the node/edge creation? >> >> Thanks >> >> Luigi >> >> >> 2016-06-11 10:18 GMT+02:00 K Major <[email protected]>: >> >>> Hello, >>> I'm trying to load a graph containing 800 000 nodes and 7 millions edges >>> using Java. >>> >>> If I do it using my laptop which has 8GB of RAM (giving 4GB to the JVM >>> via XMX4GB ) this become unfeasible : it loads correctly all the nodes but >>> it seems to be stuck after having loaded 2.5 millions nodes, after few >>> hours, it gives nothing. >>> >>> However, If I run this batch loading on another computer which has 16GB >>> of RAM (giving 10GB to the JVM using XMX10GB), the loading completes after >>> 18 mins. >>> >>> *Question : * >>> Does this mean that the bigger the graph I want to load, the more RAM I >>> would need or is there a minimum required amount of RAM to be able to load >>> all graphs ? >>> >>> >>> *Additionnal parameter used for batch loading :* >>> Java arguments: >>> -Xmx4000M >>> -Dstorage.useWAL=false >>> -Dstorage.wal.syncOnPageFlush=false >>> -Dtx.useLog=false >>> -Dstorage.diskCache.bufferSizeE=0 >>> -Xss4M >>> >>> Java Code: >>> >>> factory.declareIntent(new OIntentMassiveInsert()); >>> graphDb = factory.getNoTx(); >>> graphDb.getRawGraph().setRetainRecords(false); >>> graphDb.setUseLog(false); >>> >>> //Load Nodes >>> //Load Edges >>> >>> >>> >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
