I have a machine with 16GB RAM. Here are some numbers. I moved the shortest path test to the end for Orient because it won't run on 2.1rc3 due to a parsing bug and because it crashes 2.0.10. No connection pooling for Orient.
Since the database doesn't fit in RAM, ArangoDB's numbers are not that impressive (and actually worse for aggregation). ArangoDB 2.6 alpha 3 INFO using server address 127.0.0.1 INFO start INFO step 1/2 done INFO step 2/2 done INFO warmup done INFO executing shortest path for 19 paths INFO total paths length: 85 INFO ----------------------------------------------------------------------------- INFO ArangoDB: shortest path, 19 items INFO Total Time for 19 requests: 133919 ms INFO Average: 7048.368421052632 ms INFO ----------------------------------------------------------------------------- INFO executing neighbors for 500 elements INFO total number of neighbors found: 9102 INFO ----------------------------------------------------------------------------- INFO ArangoDB: neighbors, 500 items INFO Total Time for 500 requests: 1618 ms INFO Average: 3.236 ms INFO ----------------------------------------------------------------------------- INFO executing distinct neighbors of 1st and 2nd degree for 500 elements INFO total number of neighbors2 found: 418236 INFO ----------------------------------------------------------------------------- INFO ArangoDB: neighbors2, 500 items INFO Total Time for 500 requests: 4807 ms INFO Average: 9.614 ms INFO ----------------------------------------------------------------------------- INFO executing single read with 100000 documents INFO ----------------------------------------------------------------------------- INFO ArangoDB: single reads, 100000 items INFO Total Time for 100000 requests: 32723 ms INFO Average: 0.32723 ms INFO ----------------------------------------------------------------------------- INFO executing single write with 100000 documents INFO ----------------------------------------------------------------------------- INFO ArangoDB: single writes, 100000 items INFO Total Time for 100000 requests: 45902 ms INFO Average: 0.45902 ms INFO ----------------------------------------------------------------------------- INFO executing aggregation INFO ----------------------------------------------------------------------------- INFO ArangoDB: aggregate, 1 items INFO Total Time for 1 requests: 199743 ms INFO Average: 199743 ms INFO ----------------------------------------------------------------------------- DONE orientdb 2.1rc3 INFO using server address 127.0.0.1 INFO start INFO warmup done INFO executing neighbors for 500 elements INFO total number of neighbors found: 9102 INFO ----------------------------------------------------------------------------- INFO OrientDB: neighbors, 500 items INFO Total Time for 500 requests: 5206 ms INFO Average: 10.412 ms INFO ----------------------------------------------------------------------------- INFO executing distinct neighbors of 1st and 2nd degree for 500 elements INFO total number of neighbors2 found: 418236 INFO ----------------------------------------------------------------------------- INFO OrientDB: neighbors2, 500 items INFO Total Time for 500 requests: 80235 ms INFO Average: 160.47 ms INFO ----------------------------------------------------------------------------- INFO executing single read with 100000 documents INFO ----------------------------------------------------------------------------- INFO OrientDB: single reads, 100000 items INFO Total Time for 100000 requests: 75391 ms INFO Average: 0.75391 ms INFO ----------------------------------------------------------------------------- INFO executing single write with 100000 documents INFO ----------------------------------------------------------------------------- INFO OrientDB: single writes, 100000 items INFO Total Time for 100000 requests: 92721 ms INFO Average: 0.92721 ms INFO ----------------------------------------------------------------------------- INFO executing aggregation INFO ----------------------------------------------------------------------------- INFO OrientDB: aggregate, 1 items INFO Total Time for 1 requests: 73120 ms INFO Average: 73120 ms INFO ----------------------------------------------------------------------------- INFO executing shortest path for 19 paths ERROR OrientDB.RequestError: Error on parsing command at position #34: Invalid keyword '0].RID' Command: select shortestPath($a[0].rid, $b[0].rid, "OUT") LET $a = (select @rid from PROFILE where _key = "P714635"), $b = (select @rid from PROFILE where _key = "P117865") ------------------------------------------^ orientdb 2.0.10 INFO using server address 127.0.0.1 INFO start INFO warmup done INFO executing neighbors for 500 elements INFO total number of neighbors found: 9102 INFO ----------------------------------------------------------------------------- INFO OrientDB: neighbors, 500 items INFO Total Time for 500 requests: 1891 ms INFO Average: 3.782 ms INFO ----------------------------------------------------------------------------- INFO executing distinct neighbors of 1st and 2nd degree for 500 elements INFO total number of neighbors2 found: 418236 INFO ----------------------------------------------------------------------------- INFO OrientDB: neighbors2, 500 items INFO Total Time for 500 requests: 60942 ms INFO Average: 121.884 ms INFO ----------------------------------------------------------------------------- INFO executing single read with 100000 documents INFO ----------------------------------------------------------------------------- INFO OrientDB: single reads, 100000 items INFO Total Time for 100000 requests: 55399 ms INFO Average: 0.55399 ms INFO ----------------------------------------------------------------------------- INFO executing single write with 100000 documents INFO ----------------------------------------------------------------------------- INFO OrientDB: single writes, 100000 items INFO Total Time for 100000 requests: 96845 ms INFO Average: 0.96845 ms INFO ----------------------------------------------------------------------------- INFO executing aggregation INFO ----------------------------------------------------------------------------- INFO OrientDB: aggregate, 1 items INFO Total Time for 1 requests: 61498 ms INFO Average: 61498 ms INFO ----------------------------------------------------------------------------- INFO executing shortest path for 19 paths ERROR OrientDB.RequestError: Error on retrieving record #13:754877 (cluster: relation) On Friday, June 12, 2015 at 12:02:01 AM UTC-7, l.garulli wrote: > > Hey guys, > > I wish somebody from ArangoDB asked for help on using OrientDB properly > for such benchmark before posting results. > > But I can also understand a minor vendor tries to get attention in that > way ;-) > > We had the chance to look at the code and it doesn't use OrientDB > properly. We'll send a Pull Request to that project with our changes soon. > I hope they will update their numbers with the new ones on their web site. > > > Best Regards, > > Luca Garulli > CEO at Orient Technologies LTD > the Company behind OrientDB <http://orientdb.com> > > On 12 June 2015 at 08:39, Riccardo Tasso <[email protected] > <javascript:>> wrote: > >> Unfortunately performarces are very difficult to test, and benchmarks are >> very difficult to trust, especially if implemented by vendors. >> >> Riccardo >> >> 2015-06-12 8:33 GMT+02:00 scott molinari <[email protected] >> <javascript:>>: >> >>> For me personally, I can't believe the results. However, I'd like to see >>> some proper results for OrientDB too. >>> >>> I am sold on Orient over Arango currently, mainly due to the binary >>> interface and because it has multi-master replication and easy sharding >>> capabilities, which Arango still doesn't have and probably won't (the >>> multi-master replication that is). But, if Orient is actually that much >>> slower, that would be a no-go for our project too. >>> >>> Scott >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
