You are not supposed to set the pagesize
And leave off that cache capacity
Von meinem iPhone gesendet
> Am 28.04.2016 um 21:37 schrieb Mohammad Hossain Namaki <mhn.na...@gmail.com>:
>
> Dear all,
> Hi,
> I've wrote a java code to import "dblp" into the Neo4j. It's using the
> batchInserter to create the dataset. However, the code is stuck at
> db.shutdown() line after two days run with 20 GB memory. The current neo4j
> dataset (e.g. dblp.db) is 3.2 GB and the last modified date of files inside
> of it is for the 2 days ago (but the program didn't finish). However, when I
> download the dataset and try to run it with Neo4j Server/Java API, it cannot
> start.
>
> Could you please guide me on that?
> It seems that the number of entities to be added is 6,600,000 and number of
> relationships to be added is 11,550,000.
>
> I've attached the messages.log file and also the source code. I'm using
> neo4j.2.3.0 jar file.
>
> The summary of the source code at creating section is like this:
>
>
>
> Map<String, String> config = new HashMap<String, String>();
>
>
> config.put("dbms.pagecache.memory", "50000M");
>
>
> config.put("dbms.pagecache.pagesize", "8g");
>
>
> config.put("node_auto_indexing", "true");
>
>
> db = BatchInserters.inserter("dblp2.db", config);
>
>
> indexProvider = new LuceneBatchInserterIndexProvider(db);
>
>
> index = indexProvider.nodeIndex("dblpIndex",
> MapUtil.stringMap("type", "exact"));
>
>
> index.setCacheCapacity(KEY_PROPERTY, 500000001);
> ........
>
>
> Label label = DynamicLabel.label(author);
>
> long nodeId = db.createNode(null, label);
>
>
> ......
>
> if ((totalEntity % 50000) == 0) {
>
> index.flush();
>
> }
>
>
> .......
>
>
> Long nodeId = db.createNode(props, labels.toArray(new
> Label[labels.size()]));
>
> .......
>
> db.createRelationship(entityKeyNodeMap.get(key),
> distinctAuthors.get(author), RelTypes.WRITTEN_BY, null);
>
> ......
>
> if ((totalEntity % 50000) == 0) {
>
> index.flush();
>
> System.out.println("relationship: " + totalEntity);
>
>
> }
>
>
>
> System.out.println("indexProvider shutting down");
>
> indexProvider.shutdown();
>
>
>
> System.out.println("db shutting down");
>
> db.shutdown();
>
> System.out.println("program is finished!");
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> }
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to neo4j+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <messages.log>
> <DBLPImporter.zip>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
