Hello Andrey Lomakin, as I write the original tests that Andrey Yesyev is basing his on, I thought I need to step in with a word of explanation.
Let me start by saying your findings are correct, the test indeed inserts given amount of vertices and the a given amount of edges between the first two vertices. Generally speaking you are also right saying "this benchmark does not reproduce real test cases". However *it was never meant to be a general purpose benchmark* (please have a look at the disclaimer of my original post https://groups.google.com/forum/#!topicsearchin/orient-database/perfomance%7Csort:date%7Cspell:true/orient-database/VF_j5rGeffA). The purpose of this test was to illustrate the fact that I found OrientDB to be very slow on inserting edges. In fact getting slower and slower as the amount of edges increases. I also compared it to Neo4j just because I wasn't sure whether this is something OrientDB specific or it's due to the nature of the graph databases in general. As far as transactions are concern, my original code did not use transactions at all (at least not explicitly). According tho the docs (back then) the was supposed execute each operation instantly. I don't know (didn't have the time to examine Andrey's code) why he introduced transactions and while I agree inserting millions of documents in a single transaction is not a good idea, I just wanted to point out the original test was demonstrating the problem with no transactions at all. I'm pretty sure Andrey can easily change the code to commit data in smaller chunks but honestly speaking I don't expect huge improvements (comparing to the no transaction). As far as the structure of the data is concerned, I fail to see how can that cause performance degradation. Are you saying that if the test was to create edges between every 2 vertices for example (instead of just first 2) it would be faster? I highly doubt it. In fact I think the way the test is written should actually allow OrientDB to perform better than average as it can utilize cache and doesn't have to look for edges. Finally, I have to admin I gave up on OrientDB half a year ago (don't get me wrong, nothing personal, I just found it not to be mature enough for the project I was working on) and while I'm still trying to keep an eye on this list, I'm not fully aware of all the optimizations that have happened since then. It may me the case that the test is no longer valid for the current version or needs to be rewritten completely. If I find some spare time I will try to update my original tests to use the latest version and post some results here. Regards, Milen On Sun, Feb 9, 2014 at 8:27 PM, Andrey Lomakin <[email protected]>wrote: > Hi Andrey, > I started benchmark on my side and while it is running I investigated it. > I think that I should note that this benchmark does not reproduce real > test cases (dunno what performance data you get on other DBs). > > I mean what this benchmark does. > Lets suppose that we have to insert 1 000 000 documents vertexes and edges. > Then it creates 500 000 vertexes and then takes 2 of them, and creates 500 > 000 edges between them. > And everything in one transaction. > > So we have graph database with 499 998 unconnected vertexes and 2 vertexes > which have 500 000 edges and everything is committed in single transaction. > Did I miss something ? > > I mean that I think you do not suppose users to commit such data structure > and commit it using single transaction. > Usually data structures are way different and changes are committed in > following way users load data, change them, commit them. > > It is my personal opinion but may be you will be interested in performance > test which loads real wikipedia data by loading and committing them by > small batches ? > Also this tests uses index which is very typical for db usage. > > We used such test case so I can change and publish it as maven project and > because it is tinkerpop based you can test all dbs which you are interested > in. > Our load test does not have properties on vertexes only relations and > index by page key,but it is simple to add additional properties. > > What do you think ? > > > > On Sun, Feb 9, 2014 at 5:59 PM, Andrey Yesyev <[email protected]>wrote: > >> Please post your results! >> >> Again, any comments regarding source code are very welcome! >> >> >> On Sunday, February 9, 2014 10:50:34 AM UTC-5, Andrey Lomakin wrote: >> >>> Andrey, >>> I do not see any commits in project. https://github.com/ >>> ayesyev/graphdb-tests/commits/master >>> Did you push them ? >>> >>> >>> On Sun, Feb 9, 2014 at 5:47 PM, Andrey Lomakin <[email protected]>wrote: >>> >>>> Got it ! )) >>>> >>>> >>>> On Sun, Feb 9, 2014 at 5:44 PM, Andrey Lomakin <[email protected]>wrote: >>>> >>>>> Hi Andrey, >>>>> >>>>> Could you provide instructions how to run these tests to see statistic >>>>> results ? >>>>> >>>>> >>>>> On Sun, Feb 9, 2014 at 4:59 PM, Andrey Yesyev <[email protected]>wrote: >>>>> >>>>>> Ok, here we go! >>>>>> >>>>>> I added all Andrey's tips to the project. >>>>>> >>>>>> storage.diskCache.bufferSize set to 14336 >>>>>> >>>>>> All edges have appropriate number of properties and added this way >>>>>> >>>>>> protected OrientEdge createEdge(Vertex v1, Vertex v2) { >>>>>> Map<String, String> properties = new HashMap<String, >>>>>> String>(); >>>>>> for (int i = 0; i < numberOfProperties; i++) >>>>>> properties.put("property" + i, "value" + i); >>>>>> OrientEdge e = ((OrientVertex)v1).addEdge(null, >>>>>> (OrientVertex)v2, "E", null, properties); >>>>>> e.save(); >>>>>> >>>>>> return e; >>>>>> } >>>>>> >>>>>> Results are attached for remote and embedded (both using plocal >>>>>> storage type). >>>>>> On Monday I'll try to make my conclusions. >>>>>> >>>>>> All changes are committed to github project. >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OrientDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Andrey Lomakin. >>>>> >>>>> Orient Technologies >>>>> the Company behind OrientDB >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey Lomakin. >>>> >>>> Orient Technologies >>>> the Company behind OrientDB >>>> >>>> >>> >>> >>> -- >>> Best regards, >>> Andrey Lomakin. >>> >>> Orient Technologies >>> the Company behind OrientDB >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > > -- > Best regards, > Andrey Lomakin. > > Orient Technologies > the Company behind OrientDB > > -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- http://about.me/milen -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
