[Neo4j] Slow node creation/commits with unique constraint in 2.0.0

[Moss Prescott]

Hi,

I am experimenting with 2.0.0 and notice that it is approx. an order of 
magnitude slower to add nodes when a unique constraint is defined, versus 
an index on the same label and property. I've attached a simple program 
that demonstrates the behavior, adding increasing numbers of simple, 
unconnected nodes, with either an index or a constraint.

I'm pretty sure the index and constraint are working properly (in the sense 
that they improve query performance), based on other tests I've done.

I'm getting org.neo4j:neo4j:2.0.0 from maven central, and running on jdk 
1.7.0_40 on Fedora with no particular flags (but apparently max heap of 
3GB).

Here's the output of the test program:

indexing...
waiting...
adding 1,000 nodes...
added...
committing...
...done; 0.8s

adding constraint...
adding 1,000 nodes...
added...
committing...
...done; 3.3s

indexing...
waiting...
adding 10,000 nodes...
added...
committing...
...done; 2.9s

adding constraint...
adding 10,000 nodes...
added...
committing...
...done; 44.6s

indexing...
waiting...
adding 100,000 nodes...
added...
committing...
...done; 6.3s

adding constraint...
adding 100,000 nodes...

It runs for quite a while before even getting to the commit on the last 
transaction, at 100% CPU and with the heap climbing to about 1 GB.

Maybe 100k is too many nodes in a single transaction? But as you can see 
it's more than 10x slower even at much more modest sizes. I also tried 
adding the same number of nodes, but spread across many smaller 
transactions, and it's still much slower with the constraint.

Hopefully I'm doing something dumb here. Can anyone suggest a fix or 
confirm that this isn't working the way it should?

Thanks,
- moss

-- 
You received this message because you are subscribed to the Google Groups 
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

package demo;

import java.util.concurrent.TimeUnit;

import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Label;
import org.neo4j.graphdb.Transaction;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
import org.neo4j.graphdb.schema.IndexDefinition;
import org.neo4j.graphdb.schema.Schema;

/**
 * Demonstration of slow node creation/commits with a trivial db when a 
 * unique constraint is present.
 */
public class SlowConstraint {
	public static void main(String[] args) {
		new SlowConstraint().run(1000, false);
		new SlowConstraint().run(1000, true);
		new SlowConstraint().run(10000, false);
		new SlowConstraint().run(10000, true);
		new SlowConstraint().run(100000, false);
		new SlowConstraint().run(100000, true);
	}

	private static final String DB_PATH = "neo4j-store";
	
	public enum DemoLabel implements Label { PERSON }; 

	private GraphDatabaseService graphDb;
	private int nextId = 0;
	
	public void run(int numNodes, boolean unique) {
		// New DB every time:
		graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH + "-" + System.currentTimeMillis());

		if (unique) {
			createUniqueConstraint(DemoLabel.PERSON, "name");
		}
		else {
			createIndex(DemoLabel.PERSON, "name");
		}
		
		makeNodes(numNodes);
		
		System.out.printf("\n");
	}
	
	
	
	private void makeNodes(int numNodes) {
		System.out.printf("adding %,d nodes...\n", numNodes);
		
		long startTime = System.nanoTime();
		
		try (Transaction tx = graphDb.beginTx()) {
			for (int i = 0; i < numNodes; i++) {
				org.neo4j.graphdb.Node node = graphDb.createNode(DemoLabel.PERSON);
				node.setProperty("name", "person" + nextId++);
			}
			System.out.printf("added...\n");
			tx.success();
			System.out.printf("committing...\n");
		}
		
		double elapsed = (System.nanoTime() - startTime)/1e9;
		System.out.printf("...done; %.1fs\n", elapsed);
	}



	private void createIndex(Label label, String property) {
		IndexDefinition idx;
		
		System.out.printf("adding index...\n");
		try (Transaction tx = graphDb.beginTx()) {
			Schema schema = graphDb.schema();
			
			idx = schema.indexFor(label)
							.on(property)
							.create();
			
			tx.success();
		}
		
		System.out.printf("waiting...\n");
		try (Transaction tx = graphDb.beginTx()) {
			Schema schema = graphDb.schema();
			schema.awaitIndexOnline(idx, 10, TimeUnit.SECONDS);
			
			// no commit
		}
	}

	private void createUniqueConstraint(Label label, String property) {
		System.out.printf("adding constraint...\n");
		
		try (Transaction tx = graphDb.beginTx()) {
			Schema schema = graphDb.schema();

			schema.constraintFor(label)
					.assertPropertyIsUnique(property)
					.create();
			
			tx.success();
		}
	}

}