Hello,
I have the following scenario:
90M nodes, all the same type, with 10 brief properties
220M rels, 8 brief properties
When running any query (for example, MATCH (a)-[:`CP`]->(b) RETURN a,b
LIMIT 25), timeout occurs. The web application (/browser) shows "unknown
error", and intermittently shows "Disconnected from Neo4j. Please check if
the cord is unplugged.". Both console.log and neo4j.0.0.log show no
errors, but the messages.log file inside the graph.db folder repeated shows
errors as follows:
2014-03-17 17:02:02.973+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 1559ms [total block
time: 224.944s]
2014-03-17 17:03:15.081+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 71095ms [total block
time: 296.039s]
2014-03-17 17:03:17.108+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 1726ms [total block
time: 297.765s]
2014-03-17 17:04:27.162+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 69752ms [total block
time: 367.517s]
2014-03-17 17:04:29.507+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 1735ms [total block
time: 369.252s]
2014-03-17 17:05:41.488+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 71878ms [total block
time: 441.13s]
2014-03-17 17:05:43.339+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 1750ms [total block
time: 442.88s]
2014-03-17 17:06:53.169+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 69527ms [total block
time: 512.407s]
2014-03-17 17:06:55.057+0000 WARN [o.n.k.EmbeddedGraphDatabase]: GC
Monitor: Application threads blocked for an additional 1787ms [total block
time: 514.194s]
and then eventually timeouts show up in console.log:
Caused by: java.io.IOException: java.util.concurrent.TimeoutException: Idle
timeout expired: 70519/30000 ms
at
org.eclipse.jetty.util.BlockingCallback.block(BlockingCallback.java:103)
~[jetty-util-9.0.5.v20130815.jar:9.0.5.v20130815]
at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:662)
[jetty-server-9.0.5.v20130815.jar:9.0.5.v20130815]
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:198)
~[jetty-server-9.0.5.v20130815.jar:9.0.5.v20130815]
at
com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:307)
~[jersey-server-1.9.jar:1.9]
at
com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:134)
~[jersey-server-1.9.jar:1.9]
along with broken pipe/EofException in neo4j.log:
Caused by: org.eclipse.jetty.io.EofException
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:186)
at
org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:400)
at
org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:111)
at
org.eclipse.jetty.io.SelectorManager$ManagedSelector.processKey(SelectorManager.java:498)
at
org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:455)
at
org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:420)
... 3 more
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:89)
at sun.nio.ch.IOUtil.write(IOUtil.java:60)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:450)
at
org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:165)
I have boosted both memory settings as well as os-level io settings via
this document:
http://docs.neo4j.org/chunked/1.6.2/configuration-linux-notes.html
Here are my memory settings:
cache_type=none
use_memory_mapped_buffers=true
# 9 bytes per node
neostore.nodestore.db.mapped_memory=900M
# 33 bytes per relationships
neostore.relationshipstore.db.mapped_memory=18150M
# 38 bytes per property
neostore.propertystore.db.mapped_memory=5G
# 60 bytes per long-string block
neostore.propertystore.db.strings.mapped_memory=5G
neostore.propertystore.db.index.keys.mapped_memory=500M
neostore.propertystore.db.index.mapped_memory=500M
wrapper.java.initmemory=8192
wrapper.java.maxmemory=8192
This is run on a quad-core dual-cpu box with 48G ram, no other programs, no
other users. Other relevant information:
neo4j version: community 2.0.1
jvm: Java(TM) SE Runtime Environment (build 1.7.0-b147)
OS: centos 4.1 (Red Hat 4.1.2-54)
Are there any other settings I can change to help with this timeout issue?
It would appear that the timeout for http requests and their response is in
java code starting the jetty server, as I can't find a setting for it
anywhere. If there isn't a setting, is there a guide for importing to a
more configurable server such as tomcat, or am I on the wrong track
completely?
Thanks,
John
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
