Hi Angelo, There is nothing obviously wrong with the code you showed, but without being able to see the rest of the code it is hard to say. I noticed that you are using a SimplePointLayer to add the nodes. This will expect the nodes to contain certain properties (like 'longitude' and 'latitude'). Since you used the batch inserter to make the nodes, instead of the SimplePointLayer, it is entirely possible that you simply did not create an expected property, or named it differently.
The line in the RTreeIndex where the error occurs in 743, which looks like https://github.com/neo4j/graph-collections/blob/master/src/main/java/org/neo4j/collections/rtree/RTreeIndex.java#L743. This line seems to be setting a property on a node, in this case the BBox property (bounding box). I'm not sure why you get an error in setting a property on a node, but perhaps it is a knock-on effect from some previous missing property. To analyse more, I think we'd need to know more, the full code, sample data, etc. Or a unit test that reproduces the problem. That would be best. Regards, Craig On Thu, May 8, 2014 at 8:36 PM, Angelo Immediata <[email protected]>wrote: > Hi Michael > > I tried what you suggested; now by using the batchInserter the graph > creation process is really faster; i'm having troubles in spatila index. > Here there is my code: > > private void indexingGraph(GraphDatabaseService > graphDb,SimplePointLayermainPointsLayer > ){ > Transaction tx=graphDb.beginTx(); > StopWatch sw = new StopWatch(); > sw.start("indicizzazione spaziale"); > try{ > int j = 0; > final long[] graphIds = graphMainNodes.values; > final boolean [] states = graphMainNodes.allocated; > for (int i = 0; i < states.length; i++) > { > > > if (states[i]) > { > //tx = graphDb.beginTx(); > mainPointsLayer.add(graphDb.getNodeById(graphIds[i])); > j++; > if(i%10000 == 0) > { > tx.success(); > tx.close(); > tx = graphDb.beginTx(); > logger.info("indicizzati "+j); > } > } > } > tx.success(); > sw.stop(); > logger.info("Indicizzazione spaziale dei nodi terminata in "+sw. > getLastTaskTimeMillis()+" millisecondi; indicizzati "+j+" nodi"); > } > catch(Exception e) > { > logger.fatal("Errore nell'indicizzazione spaziale; messaggio errore: "+e. > getMessage(), e); > if( tx != null ) > { > > > tx.failure(); > } > throw e; > } > finally > { > if( tx != null ) > { > //potresti farlo 2 volte > tx.close(); > } > if( graphDb != null ) > { > graphDb.shutdown(); > } > } > } > > Basically I have a List of graph nodes ids; in my case I have around > 170000 nodes. Durign the transaction I get this kind of error: > > 20:33:24,542 FATAL [CreateBatchGraphSinkImpl] Errore nell'indicizzazione > spaziale; messaggio errore: No property record in property chain for > Node[158832,used=true,rel=1345507,prop=-1,labels=Inline(0x0:[]),light] > contained property with key 22 > java.lang.IllegalStateException: No property record in property chain for > Node[158832,used=true,rel=1345507,prop=-1,labels=Inline(0x0:[]),light] > contained property with key 22 > at > org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.findPropertyRecordContaining(NeoStoreTransaction.java:1657) > at > org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.primitiveChangeProperty(NeoStoreTransaction.java:1666) > at > org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.nodeChangeProperty(NeoStoreTransaction.java:1635) > at > org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeSetProperty(StateHandlingStatementOperations.java:537) > at > org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeSetProperty(ConstraintEnforcingEntityOperations.java:93) > at > org.neo4j.kernel.impl.api.LockingStatementOperations.nodeSetProperty(LockingStatementOperations.java:261) > at > org.neo4j.kernel.impl.api.OperationsFacade.nodeSetProperty(OperationsFacade.java:466) > at org.neo4j.kernel.impl.core.NodeProxy.setProperty(NodeProxy.java:208) > at > org.neo4j.collections.rtree.RTreeIndex.expandParentBoundingBoxAfterNewChild(RTreeIndex.java:743) > at org.neo4j.collections.rtree.RTreeIndex.addChild(RTreeIndex.java:668) > at > org.neo4j.collections.rtree.RTreeIndex.insertInLeaf(RTreeIndex.java:531) > at org.neo4j.collections.rtree.RTreeIndex.add(RTreeIndex.java:88) > at org.neo4j.gis.spatial.DefaultLayer.add(DefaultLayer.java:82) > at > it.eng.tz.pinf.graph.importer.osm.osmosis.sinkimpl.CreateBatchGraphSinkImpl.indexingGraph(CreateBatchGraphSinkImpl.java:452) > at > it.eng.tz.pinf.graph.importer.osm.osmosis.sinkimpl.CreateBatchGraphSinkImpl.release(CreateBatchGraphSinkImpl.java:369) > at org.openstreetmap.osmosis.xml.v0_6.XmlReader.run(XmlReader.java:128) > at java.lang.Thread.run(Thread.java:744) > > > > How can I solve it? > > Thank you > Angelo > > > > > > > > Il giorno giovedì 1 maggio 2014 10:44:10 UTC+2, Angelo Immediata ha > scritto: > >> Hi there >> This is my scenario: we are building a routing system by using neo4j and >> the spatial plugin. We start from the OSM file and we read this file and >> import nodes and relationships in our graph (a custom graph model) >> Now, if we don't use the batch inserter of neo4j, in order to import a >> compressed OSM file (with compressed dimension of around 140MB, and normal >> dimensions around 2GB) it takes around 3 days on a dedicated server with >> the following characteristics: CentOS 6.5 64bit, quad core, 8GB RAM; pease >> note that the most time is related to the Neo4J Nodes and relationships >> creation; in-fact if we read the same file without doing anything with >> neo4j, the file is read in around 7 minutes (i'm sure about this becouse in >> our process we first read the file in order to store the correct osm nodes >> ids and then we read again the file in order to create the neo4j graph) >> >> Obviously we need to improve the import proces so we are trying to use >> the batchInserter. So far, so good (I need to check how much it will >> perform by using the batchInserter but I guess it will be faster); so the >> first thing I did was: let's try to use the batch inserter in a simple test >> case (very similar to our code, but without modifying our code directly) >> >> I list my software versions: >> >> - Neo4j: 2.0.2 >> - Neo4jSpatial: 0.13-neo4j-2.0.1 >> - Neo4jGraphCollections: 0.7.1-neo4j-2.0.1 >> - Osmosis: 0.43.1 >> >> Since I'm using osmosis in order to read the osm file, I wrote the >> following Sink implementation: >> >> package it.graph.batch.test; >> >> >> import java.io.File; >> import java.util.HashMap; >> import java.util.Map; >> >> >> import org.neo4j.gis.spatial.SimplePointLayer; >> import org.neo4j.gis.spatial.SpatialDatabaseService; >> import org.neo4j.graphdb.GraphDatabaseService; >> import org.neo4j.unsafe.batchinsert.BatchInserter; >> import org.neo4j.unsafe.batchinsert.BatchInserters; >> import org.neo4j.unsafe.batchinsert.SpatialBatchGraphDatabaseService; >> import org.openstreetmap.osmosis.core.container.v0_6.EntityContainer; >> import org.openstreetmap.osmosis.core.domain.v0_6.Entity; >> import org.openstreetmap.osmosis.core.domain.v0_6.Node; >> import org.openstreetmap.osmosis.core.domain.v0_6.Relation; >> import org.openstreetmap.osmosis.core.domain.v0_6.Way; >> import org.openstreetmap.osmosis.core.task.v0_6.Sink; >> >> >> >> >> public class BatchInserterSinkTest implements Sink >> { >> public static final Map<String, String> NEO4J_CFG = new HashMap<String, >> String>(); >> private static File basePath = new File("/home/angelo/Scrivania/neo4j"); >> private static File dbPath = new File(basePath, "db"); >> private GraphDatabaseService graphDb; >> private BatchInserter batchInserter; >> // private BatchInserterIndexProvider batchIndexService; >> private SpatialDatabaseService spatialDb; >> private SimplePointLayer spl; >> static >> { >> NEO4J_CFG.put( "neostore.nodestore.db.mapped_memory", "100M" ); >> NEO4J_CFG.put( "neostore.relationshipstore.db.mapped_memory", >> "300M" ); >> NEO4J_CFG.put( "neostore.propertystore.db.mapped_memory", "400M" >> ); >> NEO4J_CFG.put( "neostore.propertystore.db.strings.mapped_memory", >> "800M" ); >> NEO4J_CFG.put( "neostore.propertystore.db.arrays.mapped_memory", >> "10M" ); >> NEO4J_CFG.put( "dump_configuration", "true" ); >> } >> @Override >> public void initialize(Map<String, Object> arg0) >> { >> batchInserter = BatchInserters.inserter(dbPath.getAbsolutePath(),NEO4J_CFG >> ); >> graphDb = new SpatialBatchGraphDatabaseService(batchInserter); >> spatialDb = new SpatialDatabaseService(graphDb); >> spl = spatialDb.createSimplePointLayer("testBatch", "latitudine", >> "longitudine"); >> //batchIndexService = new LuceneBatchInserterIndexProvid >> er(batchInserter); >> >> } >> >> >> @Override >> public void complete() >> { >> // TODO Auto-generated method stub >> >> >> } >> >> >> @Override >> public void release() >> { >> // TODO Auto-generated method stub >> >> >> } >> >> >> @Override >> public void process(EntityContainer ec) >> { >> Entity entity = ec.getEntity(); >> if (entity instanceof Node) { >> >> Node osmNodo = (Node)entity; >> org.neo4j.graphdb.Node graphNode = graphDb.createNode(); >> graphNode.setProperty("osmId", osmNodo.getId()); >> graphNode.setProperty("latitudine", osmNodo.getLatitude()); >> graphNode.setProperty("longitudine", osmNodo.getLongitude()); >> spl.add(graphNode); >> >> >> } else if (entity <span style="colo >> ... > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
