Hi Andrey,
Several notes.
1. According to your previous answers I suppose that you did not set
amount of memory which is consumed by disk cache (the more the better of
course) , default value is 4Gb so *you do not use 75% of your
memory*during this benchmark, amount of memory for disk cache has the
same
importance as -Xmx for Java. To set this parameter the simplest way is to
provide system property in server.sh so if you have 16 Gb of RAM then I
think you can freely use 14GB on Windows environment without any problems.
So to set this parameter set system property in server.sh
*-Dstorage.diskCache.bufferSize=14336* . After start open console and issue
command "config get storage.diskCache.bufferSize" (after connect as root of
course) to check that this parameter was set correctly.
2. Your note about bottleneck of insertion of edges with properties is not
completely true. I looked into your code.
You do following:
OrientEdge e = db.addEdge(null, v1, v2, "E");
for (int i = 0; i < numberOfProperties; i++)
e.setProperty("property" + i, "value" + i);
So you add edge at first and then properties.
In such case following steps are performed:
a. Lightweight edge is created. It means direct link between 2 vertexes.
b. Lightweight edge is removed (which means that links on each of 2
vertexes is removed), document to hold properties is created, new links are
added.
So we have to perform 3 times more operations on collection of links than
we do without properties on edges.
To avoid this either do not use lightweight edges (typically users do not
have properties on edges so it is good choice to use lightweight edges), or
modify your code as following:
Map<String, String> properties = new HashMap<String, String>();
for (int i = 0; i < numberOfProperties; i++)
properties.put("property" + i, "value" + i);
OrientEdge e = db.addEdge(null, v1, v2, "E", properties);
In such case document for edge will be created at the same time as you
create edge so we will add links to edge document at once, and will not do
add/remove/add operations instead.
3. You use following code
OrientVertex vertex = db.addVertex("OGraphVertex",null);
But you did not create schema which means that new cluster is created
during benchmark and full db flush and fsync is performed which provides
good system slowdown:
So could you either change code to the following one:
a. OrientVertex vertex = db.addVertex(null);
b. or create "OGraphVertex" class before benchmark.
And at last about your note that multi threaded test is not much faster,
sorry to agree with this but plocal storage is very new, give us several
months and we will provide multicore support
https://github.com/orientechnologies/orientdb/issues/1678 .
BTW may I ask you to run the same tests not on remote but on plocal storage
and publish difference ?
Could you change your benchmark according suggestions above and rerun test
?
On Sat, Feb 8, 2014 at 5:36 AM, Andrey Yesyev <[email protected]>wrote:
> Here is some more results.
> The main change there, I got rid of properties on edges, because I don't
> really need that in my project.
>
> Also, I figured out that
>
> db.getRawGraph().declareIntent(new OIntentMassiveInsert());
>
> actually slow down insertion rate.... What's the purpose of it then?
>
>
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | Test inserting VERTICES_AND_EDGES threads 4
> |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | test | documents |
> vertices | edges | time (ms) | avg (doc/sec) |
> | ---------------------------------------- | --------------- |
> --------------- | --------------- | -------------------- | --------------- |
> | 1000 documents with 2 fields each | 1000 |
> 500 | 500 | 37 | 27027.03 |
> | 1000 documents with 5 fields each | 1000 |
> 500 | 500 | 71 | 14084.51 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 5000 documents with 2 fields each | 5000 |
> 2500 | 2500 | 86 | 58139.53 |
> | 5000 documents with 5 fields each | 5000 |
> 2500 | 2500 | 99 | 50505.05 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 10000 documents with 2 fields each | 10000 |
> 5000 | 5000 | 149 | 67114.09 |
> | 10000 documents with 5 fields each | 10000 |
> 5000 | 5000 | 180 | 55555.56 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 50000 documents with 2 fields each | 50000 |
> 25000 | 25000 | 813 | 61500.62 |
> | 50000 documents with 5 fields each | 50000 |
> 25000 | 25000 | 806 | 62034.74 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 10000 documents with 2 fields each | 10000 |
> 5000 | 5000 | 154 | 64935.06 |
> | 10000 documents with 5 fields each | 10000 |
> 5000 | 5000 | 180 | 55555.56 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 50000 documents with 2 fields each | 50000 |
> 25000 | 25000 | 746 | 67024.13 |
> | 50000 documents with 5 fields each | 50000 |
> 25000 | 25000 | 819 | 61050.06 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 100000 documents with 2 fields each | 100000 |
> 50000 | 50000 | 1440 | 69444.44 |
> | 100000 documents with 5 fields each | 100000 |
> 50000 | 50000 | 1780 | 56179.78 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 500000 documents with 2 fields each | 500000 |
> 250000 | 250000 | 7576 | 65997.89 |
> | 500000 documents with 5 fields each | 500000 |
> 250000 | 250000 | 10560 | 47348.48 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 1000000 documents with 2 fields each | 1000000 |
> 500000 | 500000 | 16061 | 62262.62 |
> | 1000000 documents with 5 fields each | 1000000 |
> 500000 | 500000 | 21154 | 47272.38 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
>
> As you can see, the performance is way better.
> This is the results of test where vertices don't have properties and edges
> have 2 or 5 properties.
>
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | Test inserting VERTICES_AND_EDGES threads 1
> |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | test | documents |
> vertices | edges | time (ms) | avg (doc/sec) |
> | ---------------------------------------- | --------------- |
> --------------- | --------------- | -------------------- | --------------- |
> | 1000 documents with 2 fields each | 1000 |
> 500 | 500 | 287 | 3484.32 |
> | 1000 documents with 5 fields each | 1000 |
> 500 | 500 | 215 | 4651.16 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 5000 documents with 2 fields each | 5000 |
> 2500 | 2500 | 723 | 6915.63 |
> | 5000 documents with 5 fields each | 5000 |
> 2500 | 2500 | 596 | 8389.26 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 10000 documents with 2 fields each | 10000 |
> 5000 | 5000 | 977 | 10235.41 |
> | 10000 documents with 5 fields each | 10000 |
> 5000 | 5000 | 1041 | 9606.15 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 50000 documents with 2 fields each | 50000 |
> 25000 | 25000 | 12470 | 4009.62 |
> | 50000 documents with 5 fields each | 50000 |
> 25000 | 25000 | 11266 | 4438.13 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 100000 documents with 2 fields each | 100000 |
> 50000 | 50000 | 78769 | 1269.53 |
> | 100000 documents with 5 fields each | 100000 |
> 50000 | 50000 | 68145 | 1467.46 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 500000 documents with 2 fields each | 374500 |
> 250000 | 124500 | (timeout) 617106 | 606.86 |
> | 500000 documents with 5 fields each | 370100 |
> 250000 | 120100 | (timeout) 620044 | 596.89 |
>
> |-----------------------------------------------------------------------------------------------------------------------------------------|
> | 1000000 documents with 2 fields each | 614100 |
> 500000 | 114100 | (timeout) 626038 | 980.93 |
>
> The result is way worse.
> Starting with 500000 docs, test stopped by timeout, so I didn't wait for
> the whole test suit to complete.
> I think we found the bottle neck of OrientDB.
> Avoid properties in edges!
>
> Would be great to hear why edges properties are so slow!
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "OrientDB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Best regards,
Andrey Lomakin.
Orient Technologies
the Company behind OrientDB
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
