Hi guys, I don't have the history of this thread. What issues are you running into?
David On Wed, Feb 16, 2011 at 2:37 AM, Pablo Pareja <[email protected]> wrote: > Hi Mark, > > Thanks for your contribution. > I'm using a m2.4xlarge EC2 AWS instance type (*68.4 GB of memory, 26 > EC2 Compute > Units (8 virtual cores with 3.25 EC2 Compute Units each)).* > Besides I'm parsing a 17G xml file for populating the DB and normally > problems only arise when I have > already inserted a decent amount of data (around 30.000.000 nodes and > 50.000.000 relationships), that's why > I was saying it'd be kind of harsh to reproduce the same situation. > Right now I'm gonna try splitting the 17G xml file in at least 10-20 files > and doing that number of > separated batch insertions hoping it would work that way. > I'll let you all know how it goes, > > Cheers, > > Pablo > > On Wed, Feb 16, 2011 at 11:12 AM, Mark Harwood <[email protected] > >wrote: > > > Hi Pablo, > > > > >>Regarding the boiled down version of my code I guess I could prepare it > > but it's quite a big project > > > > Here's a boiled-down batch load demo I did earlier based on public > > Wikipedia data: http://code.google.com/p/graphdb-load-tester/ > > It includes what I believe to be a faster Lucene batch indexer service > > using a bloom filter and various other optimisations. > > However, the underlying Neo4J database doesn't seem to be able to cope > > with inserting these volumes of data on my available hardware and I > > don't have (and would hope not to need) 10s of gigabytes of RAM to > > throw at the problem. > > > > Cheers, > > Mark > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > -- > Pablo Pareja Tobes > LinkedIn http://www.linkedin.com/in/pabloparejatobes > Twitter http://www.twitter.com/pablopareja > > http://www.ohnosequences.com > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- David Montag Neo Technology, www.neotechnology.com Cell: 650.556.4411 [email protected] _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
