The first problem is
I don't have method db.addEdge(null, v1, v2, "E", properties);
What version of blueprints do I have to use?
For now I have blueprints-orient-graph-2.5.0-SNAPSHOT.jar
On Saturday, February 8, 2014 1:46:02 AM UTC-5, Andrey Lomakin wrote:
>
> Hi Andrey,
> Several notes.
>
> 1. According to your previous answers I suppose that you did not set
> amount of memory which is consumed by disk cache (the more the better of
> course) , default value is 4Gb so *you do not use 75% of your memory*during
> this benchmark, amount of memory for disk cache has the same
> importance as -Xmx for Java. To set this parameter the simplest way is to
> provide system property in server.sh so if you have 16 Gb of RAM then I
> think you can freely use 14GB on Windows environment without any problems.
> So to set this parameter set system property in server.sh
> *-Dstorage.diskCache.bufferSize=14336* . After start open console and
> issue command "config get storage.diskCache.bufferSize" (after connect as
> root of course) to check that this parameter was set correctly.
> 2. Your note about bottleneck of insertion of edges with properties is not
> completely true. I looked into your code.
> You do following:
>
> OrientEdge e = db.addEdge(null, v1, v2, "E");
> for (int i = 0; i < numberOfProperties; i++)
> e.setProperty("property" + i, "value" + i);
>
> So you add edge at first and then properties.
> In such case following steps are performed:
> a. Lightweight edge is created. It means direct link between 2 vertexes.
> b. Lightweight edge is removed (which means that links on each of 2
> vertexes is removed), document to hold properties is created, new links are
> added.
>
> So we have to perform 3 times more operations on collection of links than
> we do without properties on edges.
> To avoid this either do not use lightweight edges (typically users do not
> have properties on edges so it is good choice to use lightweight edges), or
> modify your code as following:
>
> Map<String, String> properties = new HashMap<String, String>();
> for (int i = 0; i < numberOfProperties; i++)
> properties.put("property" + i, "value" + i);
>
> OrientEdge e = db.addEdge(null, v1, v2, "E", properties);
>
> In such case document for edge will be created at the same time as you
> create edge so we will add links to edge document at once, and will not do
> add/remove/add operations instead.
>
> 3. You use following code
> OrientVertex vertex = db.addVertex("OGraphVertex",null);
> But you did not create schema which means that new cluster is created
> during benchmark and full db flush and fsync is performed which provides
> good system slowdown:
>
> So could you either change code to the following one:
> a. OrientVertex vertex = db.addVertex(null);
> b. or create "OGraphVertex" class before benchmark.
>
> And at last about your note that multi threaded test is not much faster,
> sorry to agree with this but plocal storage is very new, give us several
> months and we will provide multicore support
> https://github.com/orientechnologies/orientdb/issues/1678 .
>
> BTW may I ask you to run the same tests not on remote but on plocal
> storage and publish difference ?
>
> Could you change your benchmark according suggestions above and rerun
> test ?
>
>
>
>
> On Sat, Feb 8, 2014 at 5:36 AM, Andrey Yesyev
> <[email protected]<javascript:>
> > wrote:
>
>> Here is some more results.
>> The main change there, I got rid of properties on edges, because I don't
>> really need that in my project.
>>
>> Also, I figured out that
>>
>> db.getRawGraph().declareIntent(new OIntentMassiveInsert());
>>
>> actually slow down insertion rate.... What's the purpose of it then?
>>
>>
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | Test inserting VERTICES_AND_EDGES threads 4
>> |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | test | documents |
>> vertices | edges | time (ms) | avg (doc/sec) |
>> | ---------------------------------------- | --------------- |
>> --------------- | --------------- | -------------------- | --------------- |
>> | 1000 documents with 2 fields each | 1000 |
>> 500 | 500 | 37 | 27027.03 |
>> | 1000 documents with 5 fields each | 1000 |
>> 500 | 500 | 71 | 14084.51 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 5000 documents with 2 fields each | 5000 |
>> 2500 | 2500 | 86 | 58139.53 |
>> | 5000 documents with 5 fields each | 5000 |
>> 2500 | 2500 | 99 | 50505.05 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 10000 documents with 2 fields each | 10000 |
>> 5000 | 5000 | 149 | 67114.09 |
>> | 10000 documents with 5 fields each | 10000 |
>> 5000 | 5000 | 180 | 55555.56 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 50000 documents with 2 fields each | 50000 |
>> 25000 | 25000 | 813 | 61500.62 |
>> | 50000 documents with 5 fields each | 50000 |
>> 25000 | 25000 | 806 | 62034.74 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 10000 documents with 2 fields each | 10000 |
>> 5000 | 5000 | 154 | 64935.06 |
>> | 10000 documents with 5 fields each | 10000 |
>> 5000 | 5000 | 180 | 55555.56 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 50000 documents with 2 fields each | 50000 |
>> 25000 | 25000 | 746 | 67024.13 |
>> | 50000 documents with 5 fields each | 50000 |
>> 25000 | 25000 | 819 | 61050.06 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 100000 documents with 2 fields each | 100000 |
>> 50000 | 50000 | 1440 | 69444.44 |
>> | 100000 documents with 5 fields each | 100000 |
>> 50000 | 50000 | 1780 | 56179.78 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 500000 documents with 2 fields each | 500000 |
>> 250000 | 250000 | 7576 | 65997.89 |
>> | 500000 documents with 5 fields each | 500000 |
>> 250000 | 250000 | 10560 | 47348.48 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 1000000 documents with 2 fields each | 1000000 |
>> 500000 | 500000 | 16061 | 62262.62 |
>> | 1000000 documents with 5 fields each | 1000000 |
>> 500000 | 500000 | 21154 | 47272.38 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>>
>> As you can see, the performance is way better.
>> This is the results of test where vertices don't have properties and
>> edges have 2 or 5 properties.
>>
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | Test inserting VERTICES_AND_EDGES threads 1
>> |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | test | documents |
>> vertices | edges | time (ms) | avg (doc/sec) |
>> | ---------------------------------------- | --------------- |
>> --------------- | --------------- | -------------------- | --------------- |
>> | 1000 documents with 2 fields each | 1000 |
>> 500 | 500 | 287 | 3484.32 |
>> | 1000 documents with 5 fields each | 1000 |
>> 500 | 500 | 215 | 4651.16 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 5000 documents with 2 fields each | 5000 |
>> 2500 | 2500 | 723 | 6915.63 |
>> | 5000 documents with 5 fields each | 5000 |
>> 2500 | 2500 | 596 | 8389.26 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 10000 documents with 2 fields each | 10000 |
>> 5000 | 5000 | 977 | 10235.41 |
>> | 10000 documents with 5 fields each | 10000 |
>> 5000 | 5000 | 1041 | 9606.15 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 50000 documents with 2 fields each | 50000 |
>> 25000 | 25000 | 12470 | 4009.62 |
>> | 50000 documents with 5 fields each | 50000 |
>> 25000 | 25000 | 11266 | 4438.13 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 100000 documents with 2 fields each | 100000 |
>> 50000 | 50000 | 78769 | 1269.53 |
>> | 100000 documents with 5 fields each | 100000 |
>> 50000 | 50000 | 68145 | 1467.46 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 500000 documents with 2 fields each | 374500 |
>> 250000 | 124500 | (timeout) 617106 | 606.86 |
>> | 500000 documents with 5 fields each | 370100 |
>> 250000 | 120100 | (timeout) 620044 | 596.89 |
>>
>> |-----------------------------------------------------------------------------------------------------------------------------------------|
>> | 1000000 documents with 2 fields each | 614100 |
>> 500000 | 114100 | (timeout) 626038 | 980.93 |
>>
>> The result is way worse.
>> Starting with 500000 docs, test stopped by timeout, so I didn't wait for
>> the whole test suit to complete.
>> I think we found the bottle neck of OrientDB.
>> Avoid properties in edges!
>>
>> Would be great to hear why edges properties are so slow!
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> --
> Best regards,
> Andrey Lomakin.
>
> Orient Technologies
> the Company behind OrientDB
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
