2011/2/22 Shae Griffiths <[email protected]>
>
> Hi Mattias,
>
> >
> >Are you using multiple threads during batch insertion (not allowed b.t.w.)
> ?
> >
>
> It's only a single thread as far as I'm aware, unless the Lucene stuff is
> doing something funky under the covers.
>
> I was hoping it was common, and you could say "oh you've forgotten this
> line...." but my lack of googling results wasn't promising!
>
> >>
> >> I'm thinking that maybe my problem lies in the fact i'm indexing based
> on
> >> properties that may not exist on all nodes. eg. My graph is made of
> >> entities
> >> such as people and cars, a person has a first and last name, but does
> not
> >> have a registration number, and for a car it would be the reverse.
> >>
> >
> >That doesn't matter, the index can handle such data.
> >
>
> I thought this was the case, when using a non-batch approach it seemed to
> go
> for a while before it got confused!
>
> >
> >Weird... I would like to see your code to get more ideas to what it might
> >be.
> >
>
> The code to create the nodes and index them looks like this: (i'll simplify
> irrelevant bits :))
>
> // Index service and inserter are declared as:
> private static BatchInserter inserter = new BatchInserterImpl(DB_PATH);
> private static LuceneIndexBatchInserter indexService = new
> LuceneFulltextIndexBatchInserter(inserter);
>
> ...
>
> While(There_are_nodes_to_Create)
> {
> // Build up properties
> ...
> // Node is Created just before this
> long node = inserter.createNode( properties );
>
> // Each node has an ID and a Type
> indexService.index( node, ID_KEY, ID );
> indexService.index( node, TYPE_KEY, Type );
>
> // All other properties vary based on the Type of node and are read in
> from a file prior to this
> for (int j = 0; j < Number_of_properties_to_put_on_node; j ++)
> {
> if (Property_Value_isnt_null_or_empty)
> {
> indexService.index( node, Node_Property_Name, Property_Value );
> }
> }
> }
> indexService.optimize();
>
>
So with the new framework your code would look something like this (assuming
the "properties" map also contain the ID_KEY and TYPE_KEY properties):
BatchInserter db = new BatchInserterImpl( ... );
BatchInserterIndexProvider indexProvider = new
LuceneBatchInserterIndexProvider( db );
BatchInserterIndex personIndex = indexProvider.nodeIndex( "persons",
MapUtil.stringMap( "type", "exact" ) );
BatchInserterIndex carIndex = indexProvider.nodeIndex( "cars",
MapUtil.stringMap( "type", "fulltext" ) );
while ( has more nodes )
{
long node = db.createNode( properties );
personIndex.add( node, properties );
// ... or add to carIndex if it's a car
}
indexProvider.shutdown();
db.shutdown();
More information
http://wiki.neo4j.org/content/Index_Framework#Advanced_creation_and_fulltextand
http://wiki.neo4j.org/content/Index_Framework#Batch_insertion
NOTE: you'll have to recreate your indices with the new framework since it
isn't compatible with the old one.
>
> >
> >Another thing: You seem to use the old index framework. Please consider
> the
> new
> >one <http://wiki.neo4j.org/content/Index_Framework> instead. Heck, it
> might
> >even get rid of your problems also :)
> >
>
> I just had a look at that new framework and tried to copy the code (at the
> bottom, the batch insert stuff) and eclipse doesnt seem to be able to work
> out what LuceneBatchInserterIndexProvider is. Also, when looking thorough
> that documentation you sent me, I can't seem to find anything about a
> fulltext style index (or is it implicit and it works it out itself?)
>
> This indexing is my last hurdle, then once thats done, I just need to make
> a
> UI and my project is finished! YAY!
>
> Great to hear!
> Thanks,
>
> Shae
> --
> View this message in context:
> http://neo4j-user-list.438527.n3.nabble.com/Neo4j-Lucene-fulltext-index-batch-inserter-Lock-obtain-problem-tp2542866p2549074.html
> Sent from the Neo4J User List mailing list archive at Nabble.com.
> _______________________________________________
> Neo4j mailing list
> [email protected]
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Mattias Persson, [[email protected]]
Hacker, Neo Technology
www.neotechnology.com
_______________________________________________
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
