Hi, I meant storage.diskCache.bufferSize not -Xmx we do not use heap we use direct memory to avoid GC pauses. Could you run with -Dstorage.diskCache.bufferSize=37000 ?
On Thu, Sep 4, 2014 at 2:10 PM, 'Curtis Mosters' via OrientDB < [email protected]> wrote: > Here is the whole test: > > for (Vertex person : noTxGraph.getVertices("Person.ID", splitted[0])) { > for (Vertex appln : noTxGraph.getVertices("Appln.ID", splitted[1])) { > Edge edge = noTxGraph.addEdge("class:WROTE", person, appln, > "WROTE"); > edge.setProperty("appltSeqNr", splitted[2]); > edge.setProperty("intrSeqNr", splitted[3]); > } > } > > 10000.0 on 400.0 entry/sec > 20000.0 on 512.8205 entry/sec > 30000.0 on 447.7612 entry/sec > 40000.0 on 500.0 entry/sec > 50000.0 on 561.7977 entry/sec > 60000.0 on 566.0377 entry/sec > 70000.0 on 514.7059 entry/sec > 80000.0 on 444.44446 entry/sec > 90000.0 on 379.74683 entry/sec > 100000.0 on 324.67532 entry/sec > 110000.0 on 279.89822 entry/sec > 120000.0 on 244.39919 entry/sec > 130000.0 on 215.23178 entry/sec > 140000.0 on 191.25684 entry/sec > 150000.0 on 171.8213 entry/sec > > for (Vertex person : noTxGraph.getVertices("Person.ID", splitted[0])) { > for (Vertex appln : noTxGraph.getVertices("Appln.ID", splitted[1])) { > Edge edge = noTxGraph.addEdge(null, person, appln, "WROTE"); > edge.setProperty("appltSeqNr", splitted[2]); > edge.setProperty("intrSeqNr", splitted[3]); > } > } > > 10000.0 on 454.54544 entry/sec > 20000.0 on 571.4286 entry/sec > 30000.0 on 461.53845 entry/sec > 40000.0 on 512.8205 entry/sec > 50000.0 on 574.71265 entry/sec > 60000.0 on 582.5243 entry/sec > 70000.0 on 522.38806 entry/sec > 80000.0 on 449.4382 entry/sec > 90000.0 on 379.74683 entry/sec > 100000.0 on 324.67532 entry/sec > > Not working with: > > for (Vertex person : noTxGraph.getVertices("Person.ID", splitted[0])) { > for (Vertex appln : noTxGraph.getVertices( > "Appln.ID", splitted[1])) { > Edge edge = noTxGraph.addEdge(null, person > , appln, "WROTE"); > } > } > > or > > for (Vertex person : noTxGraph.getVertices("Person.ID", splitted[0])) { > for (Vertex appln : noTxGraph.getVertices( > "Appln.ID", splitted[1])) { > noTxGraph.addEdge(null, person, appln, > "WROTE"); > } > } > > > Here I get the following exception: > > com.orientechnologies.orient.core.exception.ODatabaseException: Error on > retrieving record #16:15860 (cluster: > > > > Am Donnerstag, 4. September 2014 12:56:46 UTC+2 schrieb Curtis Mosters: >> >> I set this here on the Run Configuration in Eclipse >> >> -Xmx37000M -Xms250m -XX:MaxPermSize=10000m >> >> Is that bad or nearly the same? >> >> And how can I find you setting? >> >> Btw: I have 15 GB RAM (but yeah just 3 GB are used while that process is >> running, so the Run Configuration seems to be a bad place or wrong >> parameters) >> >> >> Am Donnerstag, 4. September 2014 12:49:23 UTC+2 schrieb Andrey Lomakin: >>> >>> Hi, >>> What RAM sized do you have ? >>> Just set storage.diskCache.bufferSize to the (RAM_size_in_GB - 1.5GB) * >>> 1024. >>> >>> And you should see speed up of test if your ram size is bigger than 6GB. >>> >>> >>> On Thu, Sep 4, 2014 at 1:42 PM, 'Curtis Mosters' via OrientDB < >>> [email protected]> wrote: >>> >>>> Well I'm currently in a good state of inserting huhge amounts of data. >>>> >>>> My database has already 180mio vertices. The whole proces took round >>>> about 6 hours. >>>> >>>> My current taks is to join, or let's say create edges von 2 Classes. I >>>> do that with >>>> for (Vertex person : noTxGraph.getVertices("Person.ID", splitted[0])) { >>>> for (Vertex appln : noTxGraph.getVertices("Appln.ID", splitted[1])) >>>> { >>>> Edge edge = noTxGraph.addEdge("class:WROTE", person, appln, >>>> "WROTE"); >>>> edge.setProperty("appltSeqNr", splitted[2]); >>>> edge.setProperty("intrSeqNr", splitted[3]); >>>> } >>>> } >>>> >>>> While I had for all inserting tasks like *20k vertices/sec*. I here >>>> have *200 vertices/sec*. So 100 times sloewr >>>> >>>> The files having ~60 mio lines. >>>> >>>> So some math: >>>> >>>> - 12.000 every minute >>>> - 720.000 every hour >>>> >>>> So this would take > 3 days. >>>> >>>> Really need this till tomorrow. Hopefully someone has a performance >>>> improvement. >>>> >>>> Thank you. >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OrientDB" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Andrey Lomakin. >>> >>> Orient Technologies >>> the Company behind OrientDB >>> >>> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
