Regarding this slow import when doing it from code - I found how to improve
it (maybe ETL thread is not the right place to describe it)
My looked like:
OrientGraphNoTx graph = new OrientGraphNoTx("plocal:D:/orientdb/mydb");
// do importing of vertices - this part was ok and fast engough
// do importing of edges - this part was terrible slow.
graph.commit();
graph.shutdown();
I tried many variants - to keep vertices in map instead of reading them
from db when importing edges, but none of that was good enough.
Trick is this:
OrientGraphNoTx graph = new OrientGraphNoTx("plocal:D:/orientdb/mydb");
// do importing of vertices -
graph.commit(); // I think this is optional if I am using OrientGraphNoTx
graph.shutdown();
graph = new OrientGraphNoTx("plocal:D:/orientdb/mydb");
// do importing of edges
graph.commit(); // I think this is optional if I am using OrientGraphNoTx
graph.shutdown();
The trick is to close graph after importing of vertices and open it again
when importing edges. For some reason this is much, much faster before!
On Friday, 24 October 2014 17:20:16 UTC+2, Lvc@ wrote:
>
> Hi,
> Try Tx and commit every 1k:
>
> * tx: true,*
> * batchCommit: 1000,*
>
>
> Lvc@
>
>
> On 24 October 2014 17:18, BojanV <[email protected] <javascript:>> wrote:
>
>> If I use OrientGraphNoTx (as recommended) does begin/commit have sense at
>> all?
>>
>>
>>
>> On Friday, 24 October 2014 16:12:08 UTC+2, Enrico Risa wrote:
>>>
>>> Yes it is correct
>>>
>>> i have a similar importer. 50 edges per seconds are too low
>>> seems strange
>>>
>>> Can you try to use the OrientGraph and commit every 1000 entries?
>>>
>>> you can use graph.begin()
>>>
>>> and graph.commit()
>>>
>>> to handle the transaction
>>>
>>>
>>>
>>> 2014-10-24 15:51 GMT+02:00 BojanV <[email protected]>:
>>>
>>> Yes, I declare it in this way, I hope it is correct
>>>>
>>>> OrientGraphNoTx graph = new OrientGraphNoTx("plocal:D:/
>>>> orientdb/atlas");
>>>> graph.getRawGraph().declareIntent(new OIntentMassiveInsert());
>>>>
>>>>
>>>>
>>>> On Friday, 24 October 2014 15:49:45 UTC+2, Enrico Risa wrote:
>>>>>
>>>>> Do you use the Massive Insert intent ?
>>>>>
>>>>> 2014-10-24 15:43 GMT+02:00 BojanV <[email protected]>:
>>>>>
>>>>>> This answer is when I am working import from Javacode:
>>>>>>
>>>>>> It differs - if my vertices are all in default class then around 50
>>>>>> edges per second, If my vertices are in different classes then inserting
>>>>>> is
>>>>>> slower - around 30. If I use OrientGraphNoTx instead of OrientGraph
>>>>>> things
>>>>>> are little bit faster but still not good enough. I posted one version of
>>>>>> my
>>>>>> code a few posts above. (Note: my machine is pretty good, and I didn't
>>>>>> do
>>>>>> any additional configuring of OriendDB - I just downloaded it and used
>>>>>> it )
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Friday, 24 October 2014 15:34:51 UTC+2, Enrico Risa wrote:
>>>>>>>
>>>>>>>
>>>>>>> How many edges per seconds do you write?
>>>>>>>
>>>>>>>
>>>>>>> Il giorno venerdì 24 ottobre 2014 14:36:25 UTC+2, BojanV ha scritto:
>>>>>>>>
>>>>>>>> No comments on this?
>>>>>>>>
>>>>>>>> Is there any configuration tweaks to be done to speed up things?
>>>>>>>>
>>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "OrientDB" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>>
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "OrientDB" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
