Hi Michael,
I'm still seeing this issue on 2.1.2 following a similar batch insertion
pattern - I've reproduced approximately 5x, always occurs at the same
location:
Exception in thread "main"
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException:
PropertyRecord[1386325] not in use
at
org.neo4j.kernel.impl.nioneo.store.PropertyStore.getRecord(PropertyStore.java:423)
at
org.neo4j.kernel.impl.nioneo.store.PropertyStore.getRecord(PropertyStore.java:340)
at org.neo4j.kernel.impl.nioneo.xa.Loaders$2.load(Loaders.java:101)
at org.neo4j.kernel.impl.nioneo.xa.Loaders$2.load(Loaders.java:81)
at
org.neo4j.unsafe.batchinsert.DirectRecordAccess.getOrLoad(DirectRecordAccess.java:73)
at
org.neo4j.unsafe.batchinsert.DirectRecordAccess.getOrLoad(DirectRecordAccess.java:35)
at
org.neo4j.kernel.impl.nioneo.xa.PropertyCreator.setPrimitiveProperty(PropertyCreator.java:240)
at
org.neo4j.unsafe.batchinsert.BatchInserterImpl.setNodeProperty(BatchInserterImpl.java:309)
at
com.axiomzen.neo4j.yago.FileHandler.insertNonStringPrimitive(FileHandler.java:222)
at com.axiomzen.neo4j.yago.FileHandler.insertPrimitive(FileHandler.java:184)
at
com.axiomzen.neo4j.yago.FileHandler.handlePrimitiveTriple(FileHandler.java:164)
at com.axiomzen.neo4j.yago.FileHandler.readEntities(FileHandler.java:76)
at com.axiomzen.neo4j.yago.FileHandler.handleResource(FileHandler.java:195)
at com.axiomzen.neo4j.yago.Main.main(Main.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Was a fix pushed for this or is it in the works?
Thanks,
Ian
On Friday, April 13, 2012 12:56:20 AM UTC-7, Michael Hunger wrote:
>
> Marc,
>
> I looked at it last night and it seems to be an issue in the
> setNodeProperty() method in BatchInserter, a fix will be probably in there
> for the 1.7.GA
>
> Thanks
>
> Michael
>
> Am 13.04.2012 um 07:13 schrieb MPM:
>
> > I've found something interesting and useful that could be a workaround
> > solution to the org.neo4j.kernel.impl.nioneo.s
> > tore.InvalidRecordException: Record[1000000] not in use....
> >
> > If I set the property using the second option (in the code below), I
> > don't get the error. And I get the results that I expect. It seems
> > that it's an internal bug.
> >
> > Could anybody confirm it?
> >
> >
> > public static void setNodeProperty(BatchInserter inserter,
> > BatchInserterIndex index, long nodeId, String propertyName, Object
> > value)
> > {
> > // inserter.setNodeProperty(nodeId, propertyName,
> propertyName);
> >
> >
> inserter.getGraphDbService().getNodeById(nodeId).setProperty(propertyName,
> > value);
> > }
> >
> >
> >
> > On 12 abr, 23:15, MPM <[email protected]> wrote:
> >> After executing the code below, I got this error when I use the
> >> "BatchInserter API".
> >> The idea is create some nodes with some properties and later add more
> >> properties.
> >> After have been changing some values (setCacheCapacity, size of the
> >> loop, ... ) I got always the same message with "Record [1000000]". In
> >> fact nobody is getting the "node [1000000]".
> >>
> >> The messages.log file is empty.
> >>
> >> How could I solve this problem?
> >>
> >> Version: Community Edition 1.6 stable with a Mac
> >>
> >> Return error
> >> =========
> >>
> >> Create nodes
> >> 0
> >> 100000
> >> 200000
> >> 300000
> >> 400000
> >> Update properties - Round1
> >> Exception in thread "main"
> >> org.neo4j.kernel.impl.nioneo.store.InvalidRecordException:
> >> Record[1000000] not in use
> >> at
> >>
> org.neo4j.kernel.impl.nioneo.store.PropertyStore.getRecord(PropertyStore.ja
> va:
> >> 492)
> >> at
> >>
> org.neo4j.kernel.impl.nioneo.store.PropertyStore.getRecord(PropertyStore.ja
> va:
> >> 393)
> >> at
> >>
> org.neo4j.kernel.impl.batchinsert.BatchInserterImpl.setPrimitiveProperty(Ba
> tchInserterImpl.java:
> >> 304)
> >> at
> >>
> org.neo4j.kernel.impl.batchinsert.BatchInserterImpl.setNodeProperty(BatchIn
> serterImpl.java:
> >> 150)
> >> at TestBatchInsert.setNodeProperty(TestBatchInsert.java:100)
> >> at TestBatchInsert.main(TestBatchInsert.java:54)
> >>
> >> Sample code
> >> ============
> >>
> >> import java.io.File;
> >> import java.util.HashMap;
> >> import java.util.Map;
> >>
> >> import org.neo4j.graphdb.index.BatchInserterIndex;
> >> import org.neo4j.graphdb.index.BatchInserterIndexProvider;
> >> import org.neo4j.graphdb.index.IndexHits;
> >> import org.neo4j.helpers.collection.MapUtil;
> >> import org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider;
> >> import org.neo4j.kernel.impl.batchinsert.BatchInserter;
> >> import org.neo4j.kernel.impl.batchinsert.BatchInserterImpl;
> >>
> >> public class TestBatchInsert {
> >>
> >> public static void main(String[] args) {
> >> BatchInserter inserter;
> >> BatchInserterIndexProvider indexProvider;
> >> BatchInserterIndex index;
> >>
> >> deleteDir(new File("out/graph.db"));
> >>
> >> inserter = new BatchInserterImpl( "out/graph.db" );
> >> indexProvider = new LuceneBatchInserterIndexProvider(
> inserter );
> >>
> >> index = indexProvider.nodeIndex("index",
> MapUtil.stringMap( "type",
> >> "exact" ) );
> >> index.setCacheCapacity( "id", 1000000 );
> >> index.setCacheCapacity( "name", 1000000 );
> >>
> >> System.out.println("Create nodes");
> >> for (Long i=0L; i<500000L; i++){
> >>
> >> Map<String, Object> m = new HashMap<String,
> Object> ();
> >> m.put("__type__", "type_" + i);
> >> m.put("id", i);
> >> m.put("email", "email_" + i);
> >>
> >> addNode(inserter, index, m, m);
> >> if (i%100000 == 0) {
> >> System.out.println(i);
> >> index.flush();
> >> }
> >> }
> >> index.flush();
> >>
> >> System.out.println("Update properties - Round1");
> >> for (Long i=0L; i<500000L; i++){
> >>
> >> Long node = getNodeByPropertyIndex(index, "id",
> i);
> >> setNodeProperty(inserter, index, node,
> "new_property-A", "value-A"
> >> + i);
> >> setNodeProperty(inserter, index, node,
> "new_property-B", "value-B"
> >> + i);
> >> setNodeProperty(inserter, index, node,
> "new_property-C", "value-C"
> >> + i);
> >> setNodeProperty(inserter, index, node,
> "new_property-D", "value-D"
> >> + i);
> >>
> >> if (i%100000 == 0) {
> >> System.out.println(i);
> >> index.flush();
> >> }
> >> }
> >> index.flush();
> >>
> >> System.out.println("Update properties - Round2");
> >> for (Long i=0L; i<500000L; i++){
> >>
> >> Long node = getNodeByPropertyIndex(index, "id",
> i);
> >> setNodeProperty(inserter, index, node,
> "new_property-E", "new value-
> >> E" + i);
> >> setNodeProperty(inserter, index, node,
> "new_property-F", "new value-
> >> F" + i);
> >>
> >> if (i%100000 == 0) {
> >> System.out.println(i);
> >> index.flush();
> >> }
> >> }
> >> index.flush();
> >>
> >> for (Long i=100L; i<125L; i++){
> >> Map<String, Object> m =
> getNodeProperties(inserter, i);
> >> System.out.println(m.toString());
> >> }
> >>
> >> index.flush();
> >>
> >> // Make sure to shut down the index provider
> >> indexProvider.shutdown();
> >> inserter.shutdown();
> >> }
> >>
> >> public static long addNode(BatchInserter inserter,
> BatchInserterIndex
> >> index, Map<String, Object> nodeProperties, Map<String, Object>
> >> indexProperties ){
> >>
> >> long nodeId = inserter.createNode( nodeProperties );
> >> index.add( nodeId, indexProperties );
> >> return nodeId;
> >> }
> >>
> >> public static void setNodeProperty(BatchInserter inserter,
> >> BatchInserterIndex index, long nodeId, String propertyName, Object
> >> value)
> >> {
> >> inserter.setNodeProperty(nodeId, propertyName,
> value);
> >> }
> >>
> >> public static Long getNodeByPropertyIndex(BatchInserterIndex
> index,
> >> String propertyName, Object value){
> >>
> >> IndexHits<Long> ih = index.get(propertyName, value);;
> >> return ih.getSingle();
> >> }
> >>
> >> public static Map<String, Object>
> getNodeProperties(BatchInserter
> >> inserter, Long nodeId){
> >> return inserter.getNodeProperties(nodeId);
> >> }
> >>
> >> public static boolean deleteDir(File dir) {
> >> if (dir.isDirectory()) {
> >> String[] children = dir.list();
> >> for (int i=0; i<children.length; i++) {
> >> boolean success = deleteDir(new File(dir,
> children[i]));
> >> if (!success) {
> >> return false;
> >> }
> >> }
> >> }
> >>
> >> // The directory is now empty so delete it
> >> return dir.delete();
> >> }
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> }
>
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
