Thank you both for the quick replies. My dbs are of the following size:
FILESIZES ---------------- neostore.nodestore.db 70MB neostore.propertystore.db 583MB neostore.relationshipstore.db 6.6GB neostore.propertystore.db.strings 1 GB I played around with the props configuration, trying many different values. here is what I currently have in it. It seems like caching does not work well at all when I use the configuration. CONFIGURATION -------------------------- neostore.nodestore.db.mapped_memory=80M neostore.relationshipstore.db.mapped_memory=2500M neostore.propertystore.db.mapped_memory=500M neostore.propertystore.db.strings.mapped_memory=100M neostore.propertystore.db.arrays.mapped_memory=0M Peter, I have seen that it does work faster in Windows than in Linux but again Windows has 2x128GB SSD in RAID 0 (~300MB/s read). I have also tried running using the parameters "-Xms2000M -Xmx4000M -server" Mattias, I do want to do a BFS, finding the shortest path between two given nodes that I pass in as parameters. One thing I do not understand about caching (perhaps I am going about this entirely wrong) is how does the database stay in memory? In my Neo4jBFS.java file, I call the following commands: graphDb = new EmbeddedGraphDatabase (DB_PATH); /* do stuff */ indexService.shutdown(); graphDb.shutdown(); The program is called with the following command line: java Neo4jBFS sourceNode targetNode maxDepth (and the Xmx, Xms, server command lines as described above) Wouldn't that make it always the first instance of GraphDatabaseService? How can I get around that and have it always running in the background? If there is any code that you would like me to post to please let me know. Thank you again for the prompt replies! David. On Thu, Oct 28, 2010 at 2:54 PM, Mattias Persson <[email protected]>wrote: > Do you actually want to do a BFS (assuming that means breath first search) > and get all those paths back, or are you just testing performance? Also if > it's the first run for that GraphDatabaseService instance you're basically > testing your I/O performance on your hard drive since everything will have > to be read up into memory. Consecutive runs should be much faster. > > 2010/10/28 david lightstone <[email protected]> > > > Hi everyone, > > > > I'm running Neo4j on both Ubuntu and Windows 7 boxes. I have a dataset > with > > 200 million edges and 10 million nodes with a median branching factor of > > about 50 outgoing, directed edges/node. I'm trying to run the BFS search > on > > the data but am fairing unsuccessful in being able to do so in a timely > > fashion. I have tried to follow the advice on > > http://wiki.neo4j.org/content/Neo4j_Performance_Guide but I still have > > queries that can take up to 300 seconds or so to run. My Ubuntu box has > 6GB > > of RAM, and is running on a 7200RPM hard drive, while my windows box has > > 8GB > > RAM and is running off of SSDs (HDtune reports ~300 MB/s reads). > > > > I had also added an index for the nodes. > > > > Can anyone offer advice on why this process may be taking so long? The > CPU > > usage on both is very low (2-5%) and I'm pretty sure the whole thing is > HDD > > i/o limited, but I was wondering if there were any techniques or anything > > to > > actually get the query to go any faster? > > > > Judging by what I had read about Neo4j in descriptions I assumed that my > > data size was not too large to justify a long BFS (the paths can take up > to > > 300 seconds just 4 nodes away.) > > > > Thank you in advance. > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > -- > Mattias Persson, [[email protected]] > Hacker, Neo Technology > www.neotechnology.com > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
