Mike, great you had configuration working out! Your graph is quite big so an exhaustive walk will take some time. Still I suspect that 9h is quite long for this. Do you have the code somewhere so we could take a look at maybe get performance up a notch?
Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. 2010/3/9 Miguel Ángel Águila <[email protected]>: > Ok, > > finally the problem was that I had my neo.props bad configurated. This > is the good configuration: > > neostore.nodestore.db.mapped_memory=913M > neostore.relationshipstore.db.mapped_memory=11G > neostore.propertystore.db.mapped_memory=50M > neostore.propertystore.db.strings.mapped_memory=100M > neostore.propertystore.db.arrays.mapped_memory=0M > > Also I changed the type String for a int and the identifier for a long. > It made that the database decrease around 10 gb . > > Now, walking for all the nodes and edges spends 9 hours more or less. > > Thank you very much! > > Mike > > > > El vie, 26-02-2010 a las 18:19 +0100, Miguel Angel Aguila escribió: >> Ok, thank you. >> I've been thinking and I can change the String of node type for an integer. >> I'll change it and some other things and will try to execute another time, >> when the trials finishes I will comment you how have it gone. >> >> Mike >> >> 2010/2/26 Mattias Persson <[email protected]> >> >> > If it would be possible to have the "node type" as an integer it'd be >> > more space efficient. Also if you just need the "node type" for >> > lookups you don't need to set such property on the node, it could be >> > enough to just index it. >> > >> > With those data you provided your string property store should be 28G, >> > I'm guessing the rest is String properties on relationships? >> > >> > 2010/2/26 Miguel Ángel Águila <[email protected]>: >> > > Yes, one of them is "node type", but I need because I also use this >> > > property for indexing depend on the kind of node. >> > > >> > > >> > > >> > > El vie, 26-02-2010 a las 13:11 +0100, Mattias Persson escribió: >> > >> 2010/2/26 Miguel Ángel Águila <[email protected]>: >> > >> > I will explain me better. >> > >> > Every node has: >> > >> > - One String that have 5 chars. >> > >> > - One String that have 3 chars. >> > >> > - One String that have 4 chars. >> > >> > - One String that have 100 chars. >> > >> Is one of them any kind of "node type"? Because often you can infer a >> > >> type from its relationships or surrounding environment, making that >> > >> property redundant in a way. (nodes can often represent one or more >> > >> logical entities, hence a single type wouldn't suffice and that's why >> > >> it isn't built in to the API). >> > >> > >> > >> > Mike >> > >> > >> > >> > El vie, 26-02-2010 a las 10:05 +0100, Miguel Ángel Águila Lorente >> > >> > escribió: >> > >> >> I don't know if I'm answering your question but every node has 4 >> > pairs >> > >> >> of String, therefore 4 pairs of char(15). >> > >> >> >> > >> >> Mike >> > >> >> >> > >> >> >> > >> >> >> > >> >> El jue, 25-02-2010 a las 07:18 -0700, >> > >> >> [email protected] escribió: >> > >> >> > Yes, exactly the question. If it's a big string, it is quite >> > possible >> > >> >> > to have the file grow this large. >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > Miguel, approximately how long is the property value? >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > -------- Original Message -------- >> > >> >> > Subject: Re: [Neo] Java outof 64 GB ram >> > >> >> > From: Johan Svensson <[email protected]> >> > >> >> > Date: Thu, February 25, 2010 7:13 am >> > >> >> > To: Neo user discussions <[email protected]> >> > >> >> > If you store 322M strings (one string property/relationship), >> > were >> > >> >> > each string is 200 bytes, the string store will be 60GB+ in >> > size. >> > >> >> > How large are the strings you are storing? >> > >> >> > -Johan >> > >> >> > On Thu, Feb 25, 2010 at 2:55 PM, < >> > [email protected]> >> > >> >> > wrote: >> > >> >> > > The string propertystore file seems really, really large to >> > me, >> > >> >> > based >> > >> >> > > on the # of nodes/relationships/properties. The reason I >> > ask is >> > >> >> > that I >> > >> >> > > saw similar behavior in some of our early testing. >> > Extremely large >> > >> >> > > string propertystores file. It would be helpful to >> > understand why >> > >> >> > the >> > >> >> > > file gets that large, and whether it is related to some of >> > the >> > >> >> > memory >> > >> >> > > stress issues. >> > >> >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > -------- Original Message -------- >> > >> >> > > Subject: Re: [Neo] Java outof 64 GB ram >> > >> >> > > From: Miguel ngel_guila Lorente <[email protected]> >> > >> >> > > Date: Thu, February 25, 2010 1:57 am >> > >> >> > > To: Neo user discussions <[email protected]> >> > >> >> > > There are 322 million relationships, no 57 million >> > relationships. >> > >> >> > The >> > >> >> > > only propierty I save in the relationship is a String. In >> > this 322 >> > >> >> > > million relationships there are 3 diferents types of >> > relations. >> > >> >> > > Do you think that 13 GB is a strange number? Why? >> > >> >> > _______________________________________________ >> > >> >> > Neo mailing list >> > >> >> > [email protected] >> > >> >> > [1]https://lists.neo4j.org/mailman/listinfo/user >> > >> >> > >> > >> >> > References >> > >> >> > >> > >> >> > 1. https://lists.neo4j.org/mailman/listinfo/user >> > >> >> > _______________________________________________ >> > >> >> > Neo mailing list >> > >> >> > [email protected] >> > >> >> > https://lists.neo4j.org/mailman/listinfo/user >> > >> >> >> > >> >> >> > >> >> _______________________________________________ >> > >> >> Neo mailing list >> > >> >> [email protected] >> > >> >> https://lists.neo4j.org/mailman/listinfo/user >> > >> > >> > >> > >> > >> > _______________________________________________ >> > >> > Neo mailing list >> > >> > [email protected] >> > >> > https://lists.neo4j.org/mailman/listinfo/user >> > >> > >> > >> >> > >> >> > >> >> > > >> > > >> > > _______________________________________________ >> > > Neo mailing list >> > > [email protected] >> > > https://lists.neo4j.org/mailman/listinfo/user >> > > >> > >> > >> > >> > -- >> > Mattias Persson, [[email protected]] >> > Neo Technology, www.neotechnology.com >> > _______________________________________________ >> > Neo mailing list >> > [email protected] >> > https://lists.neo4j.org/mailman/listinfo/user >> > >> _______________________________________________ >> Neo mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
