the rels, and nodes are bigger in memory from an old manual page: http://neo4j.com/docs/1.8.3/configuration-caches.html#_object_cache <http://neo4j.com/docs/1.8.3/configuration-caches.html#_object_cache>
this is not up to date anymore but gives you an idea that's why we removed the object cache in 2 .3 Heap memory usage This table can be used to calculate how much memory the data in the object cache will occupy on a 64bit JVM: Object Size Comment Node 344 B Size for each node (not counting its relationships or properties). 48 B Object overhead. 136 B Property storage (ArrayMap 48B, HashMap 88B). 136 B Relationship storage (ArrayMap 48B, HashMap 88B). 24 B Location of first / next set of relationships. Relationship 208 B Size for each relationship (not counting its properties). 48 B Object overhead. 136 B Property storage (ArrayMap 48B, HashMap 88B). Property 116 B Size for each property of a node or relationship. 32 B Data element — allows for transactional modification and keeps track of on disk location. 48 B Entry in the hash table where it is stored. 12 B Space used in hash table, accounts for normal fill ratio. 24 B Property key index. Relationships 108 B Size for each relationship type for a node that has a relationship of that type. 48 B Collection of the relationships of this type. 48 B Entry in the hash table where it is stored. 12 B Space used in hash table, accounts for normal fill ratio. Relationships 8 B Space used by each relationship related to a particular node (both incoming and outgoing). Primitive 24 B Size of a primitive property value. String 64+B Size of a string property value. 64 + 2*len(string) B (64 bytes, plus two bytes for each character in the string). > Am 17.09.2015 um 03:19 schrieb Singleton Chiu <[email protected]>: > > Okay, when I set strong cache_type, my RAM > pagecache.mem(35g) > total disk > store size: nodes,rels, properties(32g), > > It won't trigger the GC, right? > > > On Thursday, September 17, 2015 at 8:10:01 AM UTC+8, Michael Hunger wrote: > also cache-type strong will hold all nodes and relationships that were ever > loaded in memory. > > That means you have to have enough memory to hold everything (as java > objects). > > While that is fast if you have enough memory for all data it will churn > memory and cause GC if you RAM is not enough. > > I'd probably go for cache_type=weak > > > Michael > > Am 17.09.2015 um 01:53 schrieb Singleton Chiu <[email protected] > <javascript:>>: > > My settings not right, coz server won't start, changed to : > #wrapper.java.additional=-XX:+UseConcMarkSweepGC > #wrapper.java.additional=-XX:+CMSClassUnloadingEnabled > wrapper.java.additional=-XX:+UseG1GC > > Server started, I'll go and learn something about the java gc. > > > On Thursday, September 17, 2015 at 7:37:39 AM UTC+8, Singleton Chiu wrote: > My settings right now: > > > wrapper.java.additional=-XX:+UseConcMarkSweepGC > #wrapper.java.additional=-XX:+CMSClassUnloadingEnabled > wrapper.java.additional=-XX:+UseG1GC > wrapper.java.additional=-XX:-OmitStackTraceInFastThrow > wrapper.java.additional=-XX:hashCode=5 > > > # Uncomment the following lines to enable garbage collection logging > wrapper.java.additional=-Xloggc:data/log/neo4j-gc.log > wrapper.java.additional=-XX:+PrintGCDetails > wrapper.java.additional=-XX:+PrintGCDateStamps > wrapper.java.additional=-XX:+PrintGCApplicationStoppedTime > wrapper.java.additional=-XX:+PrintPromotionFailure > wrapper.java.additional=-XX:+PrintTenuringDistribution > > > # Java Heap Size: by default the Java heap size is dynamically > # calculated based on available system resources. > # Uncomment these lines to set specific initial and maximum > # heap size in MB. > wrapper.java.initmemory=20000 > wrapper.java.maxmemory=20000 > > dbms.pagecache.memory=35g > cache_type=strong > keep_logical_logs=3 days > > > Is there anything wrong with the cache_type or keep_logical_logs setting? > I just wanna a great throughput, reduce time out moment, and hope the server > recovery from stuck. > > Thanks for your time again! > > On Wednesday, September 16, 2015 at 9:28:59 PM UTC+8, Michael Hunger wrote: > Set heap to at most 16-24G should be fine, > > And switch to G1GC with -XX:+UseG1GC > instead of CMS. > > CMS is not so good for larger heaps. > > Depending on your store file-sizes (nodes + props + rels) set the pagecache > to the sum of values, so in your case 35G > > Which edition of Neo4j are you using? > > Michael > > > > Am 16.09.2015 um 15:12 schrieb Singleton Chiu <[email protected] <>>: > > I'm using the community version, > I have searched about my situation, and turned the gc log on. > And I found that the REST API got stuck during the Full GC. > > 29184.923: [GC 24510626K->23288671K(24643840K), 0.0907810 secs] > 29185.932: [GC 24515487K->23310086K(24643840K), 0.1171710 secs] > 29186.976: [GC 24536867K->23309125K(24643840K), 0.0917860 secs] > 29187.988: [GC 24535807K->23322112K(24643840K), 0.1089120 secs] > 29189.018: [GC 24548925K->23333210K(24643840K), 0.1097290 secs] > 29190.054: [GC 24560025K->23350298K(24643840K), 0.1037740 secs] > 29204.010: [Full GC 24573732K->23319822K(24643840K), 134.0548940 secs] > 29338.275: [GC 23363153K(24643840K), 0.1082740 secs] > 29367.432: [Full GC 24643839K->23328404K(24643840K), 133.3548790 secs] > 29500.901: [GC 23366365K(24643840K), 0.1045400 secs] > 29529.261: [Full GC 24643839K->23336254K(24643840K), 134.2712890 secs] > 29663.733: [GC 23367649K(24643840K), 0.1222970 secs] > 29691.674: [Full GC 24643839K->23343854K(24643840K), 133.6376480 secs] > 29825.511: [GC 23369329K(24643840K), 0.1298200 secs] > 29854.807: [Full GC 24643839K->23351726K(24643840K), 134.3462990 secs] > 29989.364: [GC 23395476K(24643840K), 0.1952480 secs] > 30018.853: [Full GC 24643839K->23359633K(24643840K), 133.7108280 secs] > 30152.764: [GC 23395952K(24643840K), 0.1885450 secs] > 30181.939: [Full GC > > Exactly get stuck at the first Full GC. > > I thought my problem is the jvm setting, > > There are totally 96 GB mem, and my usage scenarios are lots of > node,property,rel write and read, sometimes run a heavy time consuming > cypher. > > I think the heap size is not enough, so I change my setting into: > > wrapper.java.additional=-XX:+UseConcMarkSweepGC > wrapper.java.additional=-XX:+CMSClassUnloadingEnabled > wrapper.java.additional=-XX:-OmitStackTraceInFastThrow > wrapper.java.additional=-XX:hashCode=5 > > wrapper.java.initmemory=50000 > wrapper.java.maxmemory=50000 > > dbms.pagecache.memory=30g > > Total Store Size 31.88 GiB > > Is there a guide to set these settings? > > The server now running for 6 hrs normally, with RES:26g, VIRT:61g > > If it goes Full GC continuously or REST API get stuck again, > I'll come back to you with the packed info. > > > Many Thanks! > > > On Wednesday, September 16, 2015 at 8:44:52 AM UTC+8, Michael Hunger wrote: > It should definitely recover > > do you run neo4j community or enterprise? > > what are your settings? > > perhaps you can pack together > > conf/* > data/log/* > data/graph.db/messsages.log > > for some analytics > > it can also be that the heap memory is fully utilized by the 2nd level cache > so it tries to make room by clearing memory, which can take a second or two > but then should be back > > so what you could also do is to enable gc-logging in neo4j-wrapper.conf and > then also send the gc-logs together with the other information. > > Am 16.09.2015 um 02:37 schrieb Singleton Chiu <[email protected] <>>: > > Thanks for you quick response, I have already restart the server, I need to > wait the problem situation, coz I can't re-product it. > > I have checked the py2neo log: > > [2015-09-16 07:27:19] [INFO] [28681:MainThread] [http:310 send()] > GET > http://10.181.205.198:8474/db/data/ <http://10.181.205.198:8474/db/data/> > [2015-09-16 07:27:37] [INFO] [28681:MainThread] [http:306 send()] ~ > Reconnecting (timeout) > > > Normally, it response 200 OK within 1 second. > Not only this REST API, but all encounter time out for hours, not one moment. > > And I'm sure the IP and port are certainly right, coz It have run for long > time, and after restart the neo4j server, > I get the log: > [2015-09-16 07:47:16] [INFO] [7557:MainThread] [http:310 send()] > GET > http://10.181.205.198:8474/db/data/ <http://10.181.205.198:8474/db/data/> > [2015-09-16 07:47:16] [INFO] [7557:MainThread] [http:511 __init__()] < 200 OK > [785] > > > How's that possible after server running normal for hours (have response > millions of read/write REST API normally), > and suddenly goes time out? and seems not recover to normal... > > On Wednesday, September 16, 2015 at 8:22:40 AM UTC+8, Michael Hunger wrote: > Hmm, > > 1. first of all you should use the / at the end otherwise you also measure > redirect time. > 2. you should use accept header of application/json > as you don't want to measure html generation. > > Please try that and report back, for me i does > > time curl -L http://neo.jexp.de:12474/db/data/ > <http://neo.jexp.de:12474/db/data/> > { > "extensions" : { }, > "node" : "http://neo.jexp.de:12474/db/data/node > <http://neo.jexp.de:12474/db/data/node>", > "node_index" : "http://neo.jexp.de:12474/db/data/index/node > <http://neo.jexp.de:12474/db/data/index/node>", > "relationship_index" : "http://neo.jexp.de:12474/db/data/index/relationship > <http://neo.jexp.de:12474/db/data/index/relationship>", > "extensions_info" : "http://neo.jexp.de:12474/db/data/ext > <http://neo.jexp.de:12474/db/data/ext>", > "relationship_types" : "http://neo.jexp.de:12474/db/data/relationship/types > <http://neo.jexp.de:12474/db/data/relationship/types>", > "batch" : "http://neo.jexp.de:12474/db/data/batch > <http://neo.jexp.de:12474/db/data/batch>", > "cypher" : "http://neo.jexp.de:12474/db/data/cypher > <http://neo.jexp.de:12474/db/data/cypher>", > "indexes" : "http://neo.jexp.de:12474/db/data/schema/index > <http://neo.jexp.de:12474/db/data/schema/index>", > "constraints" : "http://neo.jexp.de:12474/db/data/schema/constraint > <http://neo.jexp.de:12474/db/data/schema/constraint>", > "transaction" : "http://neo.jexp.de:12474/db/data/transaction > <http://neo.jexp.de:12474/db/data/transaction>", > "node_labels" : "http://neo.jexp.de:12474/db/data/labels > <http://neo.jexp.de:12474/db/data/labels>", > "neo4j_version" : "2.3-SNAPSHOT" > } > real 0m0.055s > user 0m0.004s > sys 0m0.004s > > > Am 16.09.2015 um 02:12 schrieb Singleton Chiu <[email protected] <>>: > > Just Now, I encounter the same situation, here's the more specific info, > I would very much appreciate for your time. > > neo4j v2.2.3 java1.7 on linux 64bit 96GB mem > > node nums > Node ID 9058830 > Property ID 80189638 > Relationship ID 37459428 > > > config: > org.neo4j.server.webserver.address=0.0.0.0 > org.neo4j.server.webserver.port=8474 > remote_shell_port=1390 > > > > Problem: REST API suddenly very slow response, situation for hours. When I > restart the server, API return to normal. > > $time wget localhost:8474/db/data > --2015-09-16 07:36:09-- http <http://localhost:8474/db/data>: > <http://localhost:8474/db/data>//localhost:8474/db/data > <http://localhost:8474/db/data> > Resolving localhost... 127.0.0.1 > ... > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
