Wow. "Neo4J is optimized for keys to find certain nodes or relationships which
normally are not more than a dozen." That's quite a surprise to me, and I hope
it not to be the case!
-----Original Message-----
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Michael Hunger
Sent: Tuesday, June 28, 2011 7:58 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Unexpected error
Hi Massimo,
still the lucene document holds an ArrayList with all fields which grows
immensely as you add millions of fields.
package org.apache.lucene.document;
public final class Document implements java.io.Serializable {
List<Fieldable> fields = new ArrayList<Fieldable>();
}
So perhaps the difference is that in Neo4j we have one Document per
node/relationship, where all keys + values are added.
Which builds up those large documents.
It is unusual for Neo4j to have a single document that contains that many
entries as you normally just index the keys to find certain nodes or
relationships
which normally are not more than a dozen. And that's why the neo4j lucene index
implementation is and will be optimized for exactly this case.
So I'd suggest as Mattias said you create your own datasource and handle the
adding there (obviously just adding node-ids + hashes to an arbitrary lucene
index.
(with one doc per node-id (or hash).
Cheers
Michael
Am 28.06.2011 um 10:21 schrieb Massimo Lusetti:
> On Mon, Jun 27, 2011 at 11:56 PM, Michael Hunger
> <michael.hun...@neotechnology.com> wrote:
>
>> Massimo,
>>
>> could you please look into the Lucene Document instance that you add all the
>> fields to?
>
> You're right... I add only the NodeId and my own hash... Which fields
> do you add?
>
>> If it also contains this ultralarge ArrayList with all the Fields ?
>
> As I said It only contains the Fields relative to the NodeId and the Hash
>
>> And which version of lucene did you use for your standalone testing?
>
> I use 3.1.0.
>
> BTW I'm testing resin and even couchdb for the index part but I don't
> like it that much cause that index phase will be out of the node4j
> Transaction and I want to isolate all that operations in a Tranaction.
> Have you hint for me on how to "extend" the node4j Transaction... is
> that extensible?
>
> Thanks
> --
> Massimo
> http://meridio.blogspot.com
> _______________________________________________
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
_______________________________________________
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
_______________________________________________
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
