Ok,
here is the final code. First of all, i walk around all the nodes and
edges, and finally I do a Traverse for storing some information.
//Query 1
public static void getNodeMaxOutDegreeNeo(final
EmbeddedGraphDatabase neo,
final IndexService indexService) {
Node idNode;
long degree;
long maxDegree = 0;
IndexHits<Node> nodes;
long idDegree;
long idMaxDegree;
Transaction tx = neo.beginTx();
try {
nodes = indexService.getNodes("TYPE", 1);
Iterator<Node> it = nodes.iterator();
Node idNodeMaxDegree=it.next();
Iterator<Relationship> relIterator =
idNodeMaxDegree.getRelationships(
NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
while (relIterator.hasNext()) {
relIterator.next();
maxDegree++;
}
long counter = 0;
while (it.hasNext()) {
if ( ++counter % 50000 == 0 ) {
tx.success();
tx.finish();
tx = neo.beginTx();
System.out.println("1.1 Porta "+counter);
}
idNode = it.next();
relIterator =
idNode.getRelationships(NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
degree = 0;
while (relIterator.hasNext()) {
relIterator.next();
degree++;
}
if (degree >= maxDegree) {
if(degree==maxDegree) {
idDegree=(Long)idNode.getProperty("ID_TITLE");
idMaxDegree=
(Long)idNodeMaxDegree.getProperty("ID_TITLE");
if(idDegree>idMaxDegree) {
idNodeMaxDegree = idNode;
}
}
else {
maxDegree = degree;
idNodeMaxDegree = idNode;
}
}
}
System.out.println("OId =
"+(Long)idNodeMaxDegree.getProperty("ID_TITLE"));
System.out.println("Title =
"+(String)idNodeMaxDegree.getProperty("NAME"));
System.out.println("#refs = "+maxDegree);
//Result:
//OId = 176122
//Title = List of years
//#refs = 5301
//Parte 1.2
tx.success();
tx.finish();
tx = neo.beginTx();
Traverser tr =
idNodeMaxDegree.traverse(Traverser.Order.BREADTH_FIRST,
StopEvaluator.END_OF_GRAPH,
ReturnableEvaluator.ALL,
MyRelationshipTypes.REF, Direction.OUTGOING);
int i=0;
Set<String> ltra = new HashSet<String>();
String id;
counter=0;
for (Node n: tr) {
if ( ++counter % 50000 == 0 ) {
tx.success();
tx.finish();
tx = neo.beginTx();
System.out.println("1.2 Porta "+counter);
}
relIterator =
n.getRelationships(NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
while (relIterator.hasNext()) {
id=Long.toString(relIterator.next().getId());
ltra.add(id);
}
if(i!=0) {
id=Long.toString(tr.currentPosition().lastRelationshipTraversed().getId());
if(ltra.contains(id)){
ltra.remove(id);
}
}
i++;
}
System.out.println("Nodes = "+i);
System.out.println("Arestes = "+(i-1));
System.out.println("Traversals= "+ltra.size());
System.out.println("Nivells =
"+tr.currentPosition().depth());
tx.success();
}
finally
{
tx.finish();
}
}
Mike.
El mar, 09-03-2010 a las 12:40 +0100, Peter Neubauer escribió:
> Mike,
> great you had configuration working out! Your graph is quite big so an
> exhaustive walk will take some time. Still I suspect that 9h is quite
> long for this. Do you have the code somewhere so we could take a look
> at maybe get performance up a notch?
>
> Cheers,
>
> /peter neubauer
>
> COO and Sales, Neo Technology
>
> GTalk: neubauer.peter
> Skype peter.neubauer
> Phone +46 704 106975
> LinkedIn http://www.linkedin.com/in/neubauer
> Twitter http://twitter.com/peterneubauer
>
> http://www.neo4j.org - Your high performance graph database.
> http://www.tinkerpop.com - Processing for Internet-scale graphs.
> http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.
>
>
>
> 2010/3/9 Miguel Ángel Águila <[email protected]>:
> > Ok,
> >
> > finally the problem was that I had my neo.props bad configurated. This
> > is the good configuration:
> >
> > neostore.nodestore.db.mapped_memory=913M
> > neostore.relationshipstore.db.mapped_memory=11G
> > neostore.propertystore.db.mapped_memory=50M
> > neostore.propertystore.db.strings.mapped_memory=100M
> > neostore.propertystore.db.arrays.mapped_memory=0M
> >
> > Also I changed the type String for a int and the identifier for a long.
> > It made that the database decrease around 10 gb .
> >
> > Now, walking for all the nodes and edges spends 9 hours more or less.
> >
> > Thank you very much!
> >
> > Mike
> >
> >
> >
> > El vie, 26-02-2010 a las 18:19 +0100, Miguel Angel Aguila escribió:
> >> Ok, thank you.
> >> I've been thinking and I can change the String of node type for an integer.
> >> I'll change it and some other things and will try to execute another time,
> >> when the trials finishes I will comment you how have it gone.
> >>
> >> Mike
> >>
> >> 2010/2/26 Mattias Persson <[email protected]>
> >>
> >> > If it would be possible to have the "node type" as an integer it'd be
> >> > more space efficient. Also if you just need the "node type" for
> >> > lookups you don't need to set such property on the node, it could be
> >> > enough to just index it.
> >> >
> >> > With those data you provided your string property store should be 28G,
> >> > I'm guessing the rest is String properties on relationships?
> >> >
> >> > 2010/2/26 Miguel Ángel Águila <[email protected]>:
> >> > > Yes, one of them is "node type", but I need because I also use this
> >> > > property for indexing depend on the kind of node.
> >> > >
> >> > >
> >> > >
> >> > > El vie, 26-02-2010 a las 13:11 +0100, Mattias Persson escribió:
> >> > >> 2010/2/26 Miguel Ángel Águila <[email protected]>:
> >> > >> > I will explain me better.
> >> > >> > Every node has:
> >> > >> > - One String that have 5 chars.
> >> > >> > - One String that have 3 chars.
> >> > >> > - One String that have 4 chars.
> >> > >> > - One String that have 100 chars.
> >> > >> Is one of them any kind of "node type"? Because often you can infer a
> >> > >> type from its relationships or surrounding environment, making that
> >> > >> property redundant in a way. (nodes can often represent one or more
> >> > >> logical entities, hence a single type wouldn't suffice and that's why
> >> > >> it isn't built in to the API).
> >> > >> >
> >> > >> > Mike
> >> > >> >
> >> > >> > El vie, 26-02-2010 a las 10:05 +0100, Miguel Ángel Águila Lorente
> >> > >> > escribió:
> >> > >> >> I don't know if I'm answering your question but every node has 4
> >> > pairs
> >> > >> >> of String, therefore 4 pairs of char(15).
> >> > >> >>
> >> > >> >> Mike
> >> > >> >>
> >> > >> >>
> >> > >> >>
> >> > >> >> El jue, 25-02-2010 a las 07:18 -0700,
> >> > >> >> [email protected] escribió:
> >> > >> >> > Yes, exactly the question. If it's a big string, it is quite
> >> > possible
> >> > >> >> > to have the file grow this large.
> >> > >> >> >
> >> > >> >> >
> >> > >> >> >
> >> > >> >> > Miguel, approximately how long is the property value?
> >> > >> >> >
> >> > >> >> >
> >> > >> >> >
> >> > >> >> > -------- Original Message --------
> >> > >> >> > Subject: Re: [Neo] Java outof 64 GB ram
> >> > >> >> > From: Johan Svensson <[email protected]>
> >> > >> >> > Date: Thu, February 25, 2010 7:13 am
> >> > >> >> > To: Neo user discussions <[email protected]>
> >> > >> >> > If you store 322M strings (one string property/relationship),
> >> > were
> >> > >> >> > each string is 200 bytes, the string store will be 60GB+ in
> >> > size.
> >> > >> >> > How large are the strings you are storing?
> >> > >> >> > -Johan
> >> > >> >> > On Thu, Feb 25, 2010 at 2:55 PM, <
> >> > [email protected]>
> >> > >> >> > wrote:
> >> > >> >> > > The string propertystore file seems really, really large
> >> > >> >> > to
> >> > me,
> >> > >> >> > based
> >> > >> >> > > on the # of nodes/relationships/properties. The reason I
> >> > ask is
> >> > >> >> > that I
> >> > >> >> > > saw similar behavior in some of our early testing.
> >> > Extremely large
> >> > >> >> > > string propertystores file. It would be helpful to
> >> > understand why
> >> > >> >> > the
> >> > >> >> > > file gets that large, and whether it is related to some of
> >> > the
> >> > >> >> > memory
> >> > >> >> > > stress issues.
> >> > >> >> > >
> >> > >> >> > >
> >> > >> >> > >
> >> > >> >> > >
> >> > >> >> > >
> >> > >> >> > > -------- Original Message --------
> >> > >> >> > > Subject: Re: [Neo] Java outof 64 GB ram
> >> > >> >> > > From: Miguel ngel_guila Lorente <[email protected]>
> >> > >> >> > > Date: Thu, February 25, 2010 1:57 am
> >> > >> >> > > To: Neo user discussions <[email protected]>
> >> > >> >> > > There are 322 million relationships, no 57 million
> >> > relationships.
> >> > >> >> > The
> >> > >> >> > > only propierty I save in the relationship is a String. In
> >> > this 322
> >> > >> >> > > million relationships there are 3 diferents types of
> >> > relations.
> >> > >> >> > > Do you think that 13 GB is a strange number? Why?
> >> > >> >> > _______________________________________________
> >> > >> >> > Neo mailing list
> >> > >> >> > [email protected]
> >> > >> >> > [1]https://lists.neo4j.org/mailman/listinfo/user
> >> > >> >> >
> >> > >> >> > References
> >> > >> >> >
> >> > >> >> > 1. https://lists.neo4j.org/mailman/listinfo/user
> >> > >> >> > _______________________________________________
> >> > >> >> > Neo mailing list
> >> > >> >> > [email protected]
> >> > >> >> > https://lists.neo4j.org/mailman/listinfo/user
> >> > >> >>
> >> > >> >>
> >> > >> >> _______________________________________________
> >> > >> >> Neo mailing list
> >> > >> >> [email protected]
> >> > >> >> https://lists.neo4j.org/mailman/listinfo/user
> >> > >> >
> >> > >> >
> >> > >> > _______________________________________________
> >> > >> > Neo mailing list
> >> > >> > [email protected]
> >> > >> > https://lists.neo4j.org/mailman/listinfo/user
> >> > >> >
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > Neo mailing list
> >> > > [email protected]
> >> > > https://lists.neo4j.org/mailman/listinfo/user
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Mattias Persson, [[email protected]]
> >> > Neo Technology, www.neotechnology.com
> >> > _______________________________________________
> >> > Neo mailing list
> >> > [email protected]
> >> > https://lists.neo4j.org/mailman/listinfo/user
> >> >
> >> _______________________________________________
> >> Neo mailing list
> >> [email protected]
> >> https://lists.neo4j.org/mailman/listinfo/user
> >
> >
> > _______________________________________________
> > Neo mailing list
> > [email protected]
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> _______________________________________________
> Neo mailing list
> [email protected]
> https://lists.neo4j.org/mailman/listinfo/user
_______________________________________________
Neo mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
