OK thank you, I decreased a little bit the number.
Until now I'm deleting without new insertion, but the size of the graph is
increasing! Is it normal? Does it depend on the block marking?
Many thanks,
Rita
Il giorno martedì 11 marzo 2014 15:02:45 UTC+1, Michael Hunger ha scritto:
>
> No, no need to restart the db, the code should look like this:
>
> Transaction tx = db.beginTx();
> int count=1;
> for (Relationship r : relationships) {
> relIndex.remove(r);
> r.delete();
> if (count++ % 50000 == 0) {
> tx.success(); tx.finish();
> tx = db.beginTx();
> }
> }
> tx.success(); tx.finish();
>
> It depends on your heap how many elements can be handled in a single tx.
> You can also lower that number to 25000
>
> Cheers,
>
> Michael
>
> ----
> (michael
> <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->(Neo4j<http://neo4j.org>
> )
> Learn Online <http://neo4j.org/learn/online_course>,
> Offline<http://www.neo4j.org/events> or
> Read a Book <http://graphdatabases.com> (in Deutsch<http://bit.ly/das-buch>
> )
> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist>
>
>
>
>
>
>
>
> Am 11.03.2014 um 14:45 schrieb Rita <[email protected] <javascript:>>:
>
> Ok, so it will not increase the space with new insertion, I hope to see
> soon this after resolving with the delete, because also if I start a new
> Transaction every 50.000 elements I got the GC error. It seems that I need
> to close and reopen also the graph to avoid this one. Does it keep all in
> memory also after tx.finish? But in this way it is too slow. Any other way
> to increase speed?
> Thanks
>
> Rita
>
> Il giorno martedì 11 marzo 2014 12:58:52 UTC+1, Michael Hunger ha scritto:
>>
>> Existing node and relationship records that are freed will of course be
>> reused (but only after a restart).
>>
>> So if you have a sliding window and delete nodes, then restart your db
>> before inserting the new window and it will use the existing record blocks
>> on disk.
>>
>> Cheers,
>>
>> Michael
>>
>> ----
>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->(
>> Neo4j <http://neo4j.org/>)
>> Learn Online <http://neo4j.org/learn/online_course>,
>> Offline<http://www.neo4j.org/events> or
>> Read a Book <http://graphdatabases.com/> (in Deutsch<http://bit.ly/das-buch>
>> )
>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist>
>>
>>
>>
>>
>>
>>
>>
>> Am 11.03.2014 um 12:10 schrieb Rita <[email protected]>:
>>
>> Thank you for the reply.
>> Really cannot exist a way to reduce the disk space? I need it. So how
>> could I do?
>> I cannot reimport the data from the beginning because I have to populate
>> the graph like a sliding window, I need to remove the older data l and then
>> import newer ones.
>> Does exist some way to "optimize" the graph and index space after those
>> deletes?
>> Yes I have an index on relationships, so I'm deleting them both from the
>> index and the graph.
>>
>> Thanks.
>>
>> Rita
>>
>>
>> Il giorno martedì 11 marzo 2014 11:34:54 UTC+1, Michael Hunger ha scritto:
>>>
>>> Neo4j is block based, it will only mark blocks as not in use but won't
>>> release them on disk. So your deletion of relationships won't make the
>>> database smaller on disk.
>>>
>>> You're probably faster reimporting the data with the batch-inserter.
>>>
>>> Did you actually index the relationships?
>>>
>>> you have to batch your transactions around your delete operations and
>>> start a new Transaction every 50.000 elements.
>>>
>>> Otherwise you will run into GC issues which explain the slowness and GC
>>> overhead.
>>>
>>>
>>> Cheers,
>>>
>>> Michael
>>>
>>> ----
>>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->(
>>> Neo4j <http://neo4j.org/>)
>>> Learn Online <http://neo4j.org/learn/online_course>,
>>> Offline<http://www.neo4j.org/events> or
>>> Read a Book <http://graphdatabases.com/> (in Deutsch<http://bit.ly/das-buch>
>>> )
>>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Am 11.03.2014 um 11:23 schrieb Rita <[email protected]>:
>>>
>>> Hi everybody
>>>
>>> I have populated my graph with nodes and relationships with the
>>> BatchInserter. Now I need to delete a part of relationships, to reduce the
>>> disk space of the graph. I have some problem. I am using the embedded
>>> version of Neo4j with Java.
>>>
>>> I am using the instruction indexRel.remove(rel) to delete relationships
>>> from the index, and it is ok.
>>>
>>> Instead I am using r.delete() inside transaction to delete them from the
>>> graph. This is very slow, does exist another method instead of the
>>> transactional one? I need also to close and reopen the graph to do not have
>>> the error: GC overhead limit exeeded. Someone have a suggestion, please?
>>>
>>> Then, why after deleting relationships, the size of the graph on disk do
>>> not decrease?
>>>
>>> Thank you very much in advance.
>>>
>>> Rita
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Neo4j" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Neo4j" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
