Re: [Neo] Java outof 64 GB ram
I see that you're using an older version of Neo4j. Have you tried using 1.0?
http://components.neo4j.org/neo4j-kernel
http://neo4j.org/download
Maven
groupId: org.neo4j
artifactId: neo4j-kernel
version: 1.0
...and at first glance I don't see any direct problem with your code.
2010/2/23 Miguel Ángel Águila magu...@ac.upc.edu:
Hello,
I'm doing a code to walk around all the nodes and get the node with the
maximum out degree. The main problem is that there are around 57 million
of nodes and 322 million of edges in the neo database, and using a 64 GB
RAM is insufficient to execute the program. It is the code:
Can anyone see what I'm doing wrong? I don't understand.
Thanks.
Mike
public static void getNodeMaxOutDegreeNeo(final NeoService neo,
final IndexService indexService) {
Node idNode;
long degree;
long maxDegree = 0;
IndexHitsNode nodes;
long idDegree;
long idMaxDegree;
Transaction tx = neo.beginTx();
try {
nodes = indexService.getNodes(TYPE, titles);
IteratorNode it = nodes.iterator();
Node idNodeMaxDegree=it.next();
IteratorRelationship relIterator =
idNodeMaxDegree.getRelationships(
NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
while (relIterator.hasNext()) {
relIterator.next();
maxDegree++;
}
long counter = 0;
while (it.hasNext()) {
if ( ++counter % 5 == 0 ) {
tx.success();
tx.finish();
tx = neo.beginTx();
}
idNode = it.next();
relIterator =
idNode.getRelationships(NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
degree = 0;
while (relIterator.hasNext()) {
relIterator.next();
degree++;
}
if (degree = maxDegree) {
if(degree==maxDegree) {
idDegree=
Long.getLong((String)idNode.getProperty(ID_TITLE));
idMaxDegree=
Long.getLong((String)idNodeMaxDegree.getProperty(ID_TITLE));
if(idDegreeidMaxDegree) {
idNodeMaxDegree = idNode;
}
}
else {
maxDegree = degree;
idNodeMaxDegree = idNode;
}
}
}
System.out.println(OId =
+(String)idNodeMaxDegree.getProperty(ID_TITLE));
System.out.println(Title =
+(String)idNodeMaxDegree.getProperty(NAME));
System.out.println(#refs = +maxDegree);
tx.success();
}
finally
{
tx.finish();
}
}
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Java outof 64 GB ram
One more thing... how much heap have you given the JVM? You control
how much RAM your JVM gets with the -Xmx option to the java command.
F.ex:
java -Xmx10G -cp .
Please see more information regarding performance and tweaking at
http://wiki.neo4j.org/content/Neo4j_Performance_Guide
2010/2/23 Mattias Persson matt...@neotechnology.com:
I see that you're using an older version of Neo4j. Have you tried using 1.0?
http://components.neo4j.org/neo4j-kernel
http://neo4j.org/download
Maven
groupId: org.neo4j
artifactId: neo4j-kernel
version: 1.0
...and at first glance I don't see any direct problem with your code.
2010/2/23 Miguel Ángel Águila magu...@ac.upc.edu:
Hello,
I'm doing a code to walk around all the nodes and get the node with the
maximum out degree. The main problem is that there are around 57 million
of nodes and 322 million of edges in the neo database, and using a 64 GB
RAM is insufficient to execute the program. It is the code:
Can anyone see what I'm doing wrong? I don't understand.
Thanks.
Mike
public static void getNodeMaxOutDegreeNeo(final NeoService neo,
final IndexService indexService) {
Node idNode;
long degree;
long maxDegree = 0;
IndexHitsNode nodes;
long idDegree;
long idMaxDegree;
Transaction tx = neo.beginTx();
try {
nodes = indexService.getNodes(TYPE, titles);
IteratorNode it = nodes.iterator();
Node idNodeMaxDegree=it.next();
IteratorRelationship relIterator =
idNodeMaxDegree.getRelationships(
NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
while (relIterator.hasNext()) {
relIterator.next();
maxDegree++;
}
long counter = 0;
while (it.hasNext()) {
if ( ++counter % 5 == 0 ) {
tx.success();
tx.finish();
tx = neo.beginTx();
}
idNode = it.next();
relIterator =
idNode.getRelationships(NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
degree = 0;
while (relIterator.hasNext()) {
relIterator.next();
degree++;
}
if (degree = maxDegree) {
if(degree==maxDegree) {
idDegree=
Long.getLong((String)idNode.getProperty(ID_TITLE));
idMaxDegree=
Long.getLong((String)idNodeMaxDegree.getProperty(ID_TITLE));
if(idDegreeidMaxDegree) {
idNodeMaxDegree = idNode;
}
}
else {
maxDegree = degree;
idNodeMaxDegree = idNode;
}
}
}
System.out.println(OId =
+(String)idNodeMaxDegree.getProperty(ID_TITLE));
System.out.println(Title =
+(String)idNodeMaxDegree.getProperty(NAME));
System.out.println(#refs = +maxDegree);
tx.success();
}
finally
{
tx.finish();
}
}
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
--
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo] getRelationships(RelationshipType... types)
Hi, I am newbie to neo4j and I am enjoying it. I have started to create nodes, relationships and assigning properties to the nodes etc. I've got a problem in accessing relationships of a node. If 'knows = DynamicRelationshipType.withName(KNOWS)' is a relation defined. When a relationship 'knows' is formed from node1 to node2, I should be able to access relationship of node1 by one of node1.getRelationships() or node1.getRelationships(knows, direction) or node1.getRelationships(direction) or node1.getRelationships(knows) methods. There is no problem with the first three, but the last method throws an error. Error is: NameError: no getRelationships with arguments matching [class org.neo4j.graphdb.DynamicRelationshipType] on object ##Class:01xf38cf0:0x1aa0a15 From the documentation of node1.getRelationships(RelationshipType... types), I sense that 'types' is a little different from 'knows' . Please help me understand how types needs to be passed to the method node1.getRelationships(RelationshipType... types). Thanks, T.Sumanth ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Java outof 64 GB ram
Yes I give 64 GB and only 60 GB in diferent executions.
2010/2/23 Mattias Persson matt...@neotechnology.com
One more thing... how much heap have you given the JVM? You control
how much RAM your JVM gets with the -Xmx option to the java command.
F.ex:
java -Xmx10G -cp .
Please see more information regarding performance and tweaking at
http://wiki.neo4j.org/content/Neo4j_Performance_Guide
2010/2/23 Mattias Persson matt...@neotechnology.com:
I see that you're using an older version of Neo4j. Have you tried using
1.0?
http://components.neo4j.org/neo4j-kernel
http://neo4j.org/download
Maven
groupId: org.neo4j
artifactId: neo4j-kernel
version: 1.0
...and at first glance I don't see any direct problem with your code.
2010/2/23 Miguel Ángel Águila magu...@ac.upc.edu:
Hello,
I'm doing a code to walk around all the nodes and get the node with the
maximum out degree. The main problem is that there are around 57 million
of nodes and 322 million of edges in the neo database, and using a 64 GB
RAM is insufficient to execute the program. It is the code:
Can anyone see what I'm doing wrong? I don't understand.
Thanks.
Mike
public static void getNodeMaxOutDegreeNeo(final NeoService neo,
final IndexService indexService) {
Node idNode;
long degree;
long maxDegree = 0;
IndexHitsNode nodes;
long idDegree;
long idMaxDegree;
Transaction tx = neo.beginTx();
try {
nodes = indexService.getNodes(TYPE, titles);
IteratorNode it = nodes.iterator();
Node idNodeMaxDegree=it.next();
IteratorRelationship relIterator =
idNodeMaxDegree.getRelationships(
NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
while (relIterator.hasNext()) {
relIterator.next();
maxDegree++;
}
long counter = 0;
while (it.hasNext()) {
if ( ++counter % 5 == 0 ) {
tx.success();
tx.finish();
tx = neo.beginTx();
}
idNode = it.next();
relIterator =
idNode.getRelationships(NeoDataBase.MyRelationshipTypes.REF,
Direction.OUTGOING).iterator();
degree = 0;
while (relIterator.hasNext()) {
relIterator.next();
degree++;
}
if (degree = maxDegree) {
if(degree==maxDegree) {
idDegree=
Long.getLong((String)idNode.getProperty(ID_TITLE));
idMaxDegree=
Long.getLong((String)idNodeMaxDegree.getProperty(ID_TITLE));
if(idDegreeidMaxDegree) {
idNodeMaxDegree = idNode;
}
}
else {
maxDegree = degree;
idNodeMaxDegree = idNode;
}
}
}
System.out.println(OId =
+(String)idNodeMaxDegree.getProperty(ID_TITLE));
System.out.println(Title =
+(String)idNodeMaxDegree.getProperty(NAME));
System.out.println(#refs = +maxDegree);
tx.success();
}
finally
{
tx.finish();
}
}
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
--
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo] Reset traverser iterator
Hi,
I was wondering if it is possible to reset a traverser iterator.
Looking at the Traverser interface comment it says...
// Doc: especially remove() thing
/**
* Returns an {...@link Iterator} representing the traversal of the graph.
The
* iteration is completely lazy in that it will only traverse one step
(to
* the next hit) for every call to {...@code hasNext()}/{...@code next()}.
*
* Consecutive calls to this method will return the same instance.
*
* @return An iterator for this traverser
*/
// *TODO completely resolve issues regarding this (Iterable/Iterator
...)*
*// Doc: does it create a new iterator or reuse the existing one? This
is*
*// very important! It must be re-use, how else would currentPosition()*
*// make sense?*
public IteratorNode iterator();
thanks
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] IllegalStateException meaning
I just got this same problem, and was able to defeat by upping heap size. It was very strange - does Transaction#finish do some non-blocking work? Disclaimer: I'm using trunk neo-kernel from 2/10. Thanks, Adam On Mon, Jun 1, 2009 at 1:31 PM, Johan Svensson jo...@neotechnology.com wrote: Great that it seems to work. Please do get back to me if you run into any problems. -Johan On Mon, Jun 1, 2009 at 7:06 PM, Rob Challen rjchal...@gmail.com wrote: Thanks Johan, I increased a whole lot of memory limits and swapped my jre over from IBM Java 64 bit to a sun 32 bit. Because it was happening after 2 hours into a load I didn't get to properly debug but so far the problem seems to have disappeared, and I can't reproduce it now. I think it may be a symptom of another problem somewhere else. Will get back to you if I can find anything more. Rob. On Sun, May 31, 2009 at 3:21 PM, Johan Svensson jo...@neotechnology.comwrote: Hi Rob, I've seen this problem once before but could not reproduce the behavior once I tried. Could you check if tx.success() is called before tx.finish()? Also, could you check in the stacktrace for the source of the nested IllegalStateException. -Johan On Sun, May 31, 2009 at 12:12 AM, Rob Challen rjchal...@gmail.com wrote: I am getting the following exception when I call a Transaction.finish(). It occurs during the load of a large medical terminology dataset, and prior to this error the load progress was found to be slowing. I can;t really figure out what the exception is telling me, or where to start looking to figure out the cause. If anyone can give me any pointers to explain the meaning of the error in neo4j I would be really grateful. The .finish() is in a loop that completes without any problem up to this point. I cannot completely rule out bad input - but finding that in a 1.5M nodes graph would be a challenge in itself. Exception in thread main java.lang.RuntimeException: java.lang.IllegalStateException: Tx status is: STATUS_COMMITING at org.neo4j.api.core.EmbeddedNeo$TransactionImpl.finish(EmbeddedNeo.java:383) ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] IllegalStateException meaning
I think it would be valuable to understand why the memory requirements are so large and how best to manage these types of situations in addition to increasing the heap, since it seems that in some cases this merely delays the issue. Is there any internal instrumentation on Neo memory usage that could be used to help tune/tweak the settings? If not, would it make sense to add a couple of MBeans for this type of information? Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Adam Rabung Sent: Tuesday, February 23, 2010 2:15 PM To: Neo user discussions Subject: Re: [Neo] IllegalStateException meaning I just got this same problem, and was able to defeat by upping heap size. It was very strange - does Transaction#finish do some non-blocking work? Disclaimer: I'm using trunk neo-kernel from 2/10. Thanks, Adam On Mon, Jun 1, 2009 at 1:31 PM, Johan Svensson jo...@neotechnology.com wrote: Great that it seems to work. Please do get back to me if you run into any problems. -Johan On Mon, Jun 1, 2009 at 7:06 PM, Rob Challen rjchal...@gmail.com wrote: Thanks Johan, I increased a whole lot of memory limits and swapped my jre over from IBM Java 64 bit to a sun 32 bit. Because it was happening after 2 hours into a load I didn't get to properly debug but so far the problem seems to have disappeared, and I can't reproduce it now. I think it may be a symptom of another problem somewhere else. Will get back to you if I can find anything more. Rob. On Sun, May 31, 2009 at 3:21 PM, Johan Svensson jo...@neotechnology.comwrote: Hi Rob, I've seen this problem once before but could not reproduce the behavior once I tried. Could you check if tx.success() is called before tx.finish()? Also, could you check in the stacktrace for the source of the nested IllegalStateException. -Johan On Sun, May 31, 2009 at 12:12 AM, Rob Challen rjchal...@gmail.com wrote: I am getting the following exception when I call a Transaction.finish(). It occurs during the load of a large medical terminology dataset, and prior to this error the load progress was found to be slowing. I can;t really figure out what the exception is telling me, or where to start looking to figure out the cause. If anyone can give me any pointers to explain the meaning of the error in neo4j I would be really grateful. The .finish() is in a loop that completes without any problem up to this point. I cannot completely rule out bad input - but finding that in a 1.5M nodes graph would be a challenge in itself. Exception in thread main java.lang.RuntimeException: java.lang.IllegalStateException: Tx status is: STATUS_COMMITING at org.neo4j.api.core.EmbeddedNeo$TransactionImpl.finish(EmbeddedNeo.java:383) ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Transaction/Exception question
If an exception cause a block of code to exit prior to a transaction being
finish()ed, will it be implicitly finished in garbage collection?
Here's the situation I want to try to implement : I would like to propagate
the exception thrown by doSomething() to the caller of myFunc(), but also
ensure that the transaction is properly failed and the transaction is
properly finished, if necessary:
public void myFunc() throws Exception {
Transaction txneo.beginTx();
try {
doSomething(); // This could throw an
exception
}
catch(Exception ex) {
tx.failure();
throw new Exception(An error has occurred:
+ ex.getMessage);
}
finally{
tx.finish();
}
}
I think I'm just not thinking clearly since I've been coding for 24 of the
last 36 hours, so I wanted to see if I'm too tired to figure out what should
be an easy exception handling scenario.
Thanks
Rick
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo] Managing nodes with same values for a property
Hi, A clarification regarding the properties of nodes. If I am not wrong two nodes can have same value for similar property. For example: node1.setProperty(name, Sumanth), node2.setProperty(name, Sumanth) is valid. node1 and node2 have same value Sumanth for the property name. My doubt is, how do we distinguish these nodes? Do we need to check if there already exists any node with the similar property value in Lucene(if one uses lucene) before creating? Thank you, T.Sumanth ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
