What I meant the server stops responding to my application's request was my application received a NoHttpResponseException. This is the output in the Eclipse console:
-Uploading /doc/test/ont/Ontology1320789957941.owl to Neo4J... Nov 10, 2011 3:46:52 PM org.restlet.ext.httpclient.HttpClientHelper start INFO: Starting the Apache HTTP client Root node of ontology Ontology1320789957941.owl: http://localhost:7474/db/data/index/node/my_index/name/Ontology1320789957941 Find node: http://localhost:7474/db/data/index/node/my_index/name/super_node Processing triple -http://localhost:7474/db/data/node/1,CONTAINS,http://localhost:7474/db/data/node/147 Find node: http://localhost:7474/db/data/index/node/my_index/name/lowEnergy Nov 10, 2011 3:47:22 PM org.restlet.ext.httpclient.internal.HttpMethodCall sendRequest WARNING: An error occurred during the communication with the remote HTTP server. org.apache.http.NoHttpResponseException: The target server failed to respond at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:101) If I upload only a few ontologies (no problems) and the log/neo4j.x.x.log showed the server processed the lowEnergy node : Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: REQUEST /db/data/node/1/relationships/out/CONTAINS on org.mortbay.jetty.HttpConnection@237dc815 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: sessionManager=org.mortbay.jetty.servlet.HashSessionManager@375e293a Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: session=null Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet=org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: chain=org.neo4j.server.statistic.StatisticFilter->org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet holder=org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call filter org.neo4j.server.statistic.StatisticFilter Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call servlet org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: RESPONSE /db/data/node/1/relationships/out/CONTAINS 200] Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: REQUEST /db/data/node/1/relationships on org.mortbay.jetty.HttpConnection@237dc815 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: sessionManager=org.mortbay.jetty.servlet.HashSessionManager@375e293a Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: session=null Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet=org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: chain=org.neo4j.server.statistic.StatisticFilter->org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet holder=org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call filter org.neo4j.server.statistic.StatisticFilter Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call servlet org.neo4j.server.web.NeoServletContainer-1130213695 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: RESPONSE /db/data/node/1/relationships 201 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: REQUEST /db/manage/server/monitor/fetch/1320942809 on org.mortbay.jetty.HttpConnection@66e90097 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: Got Session ID 9fdtr96dq71c1iqw1b8416rpc from cookie Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: sessionManager=org.mortbay.jetty.servlet.HashSessionManager@375e293a Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: session=null Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet=org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: chain=org.neo4j.server.statistic.StatisticFilter->org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: servlet holder=org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call filter org.neo4j.server.statistic.StatisticFilter Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: call servlet org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:52 PM org.mortbay.log.Slf4jLog debug FINE: RESPONSE /db/manage/server/monitor/fetch/1320942809 200 Nov 10, 2011 3:46:52 PM DefaultMBeanServerInterceptor getAttribute FINER: Attribute= HeapMemoryUsage, obj= java.lang:type=Memory Nov 10, 2011 3:46:52 PM Repository retrieve FINER: name=java.lang:type=Memory Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: REQUEST /db/manage/server/monitor/fetch/1320942588 on org.mortbay.jetty.HttpConnection@66e90097 Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: Got Session ID 9fdtr96dq71c1iqw1b8416rpc from cookie Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: sessionManager=org.mortbay.jetty.servlet.HashSessionManager@375e293a Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: session=null Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: servlet=org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: chain=org.neo4j.server.statistic.StatisticFilter->org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: servlet holder=org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: call filter org.neo4j.server.statistic.StatisticFilter Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: call servlet org.neo4j.server.web.NeoServletContainer-1301864188 Nov 10, 2011 3:46:53 PM org.mortbay.log.Slf4jLog debug FINE: RESPONSE /db/manage/server/monitor/fetch/1320942588 200 and the server displays a request for next node in my ontology. But when I upload many ontology then when the problem happened the server keeps displaying blocks of request for fetch like FINE: REQUEST /db/manage/server/monitor/fetch/xxxxxxxxxxxx on org.mortbay.jetty.HttpConnection@66e90097 Sorry for this long mail. I have no clue what causes the problem. Thank you very much, Regards ________________________________ From: David Montag <[email protected]> To: Neo4j user discussions <[email protected]> Sent: Thursday, November 10, 2011 3:31 PM Subject: Re: [Neo4j] Neo4j REST server's log files Can you describe how you see that it "stops"? Because the thread dump isn't showing anything of significance running. David On Thu, Nov 10, 2011 at 3:28 PM, andrew ton <[email protected]> wrote: > > > Hi David, > > The total size of files stored into the db when the problem happened was > only 800KB. My RAM is 4G. > I ran the kill command when the problem happened because it just stopped > and I did not notice slowness. > > I will increase the heap to 512MB and test again. > > Thanks, > Andrew > > > ________________________________ > From: David Montag <[email protected]> > To: Neo4j user discussions <[email protected]> > Sent: Thursday, November 10, 2011 3:00 PM > Subject: Re: [Neo4j] Neo4j REST server's log files > > Hi Andrew, > > I don't see anything running in that thread dump. No threads are processing > requests. Are you sure it's taking a long time, or is it maybe finished? > Try capturing the thread dump exactly when you experience slowness. I'd > also suggest bumping the initial heap to at least 512MB or something like > that. How much RAM do you have? How much data are you inserting? > > Thanks, > David > > On Thu, Nov 10, 2011 at 2:56 PM, andrew ton <[email protected]> wrote: > > > > > > > Hi David, > > > > I increased the memory settings in neo4j-wrapper.conf as below > > # Initial Java Heap Size (in MB) > > wrapper.java.initmemory=30 > > > > # Maximum Java Heap Size (in MB) > > wrapper.java.maxmemory=1024 > > > > However the same problem is still happening. I attach the thread dump to > > this mail. I appreciate it if you tell me what's wrong based on the > thread > > dump. > > > > Thank you! > > > > > > ________________________________ > > From: David Montag <[email protected]> > > To: Neo4j user discussions <[email protected]> > > Sent: Thursday, November 10, 2011 11:39 AM > > Subject: Re: [Neo4j] Neo4j REST server's log files > > > > Hi Andrew, > > > > Good to hear that you got the logging sorted out. > > > > Regarding the actual issues, it sounds like you're describing two > different > > things. One is that you index ontology nodes in one global index and > > therefore run into conflicts. The other is that the upload appears to > stall > > for some reason. > > > > Regarding the global index, did you intend to design the system that way, > > or do you really want a separate index for each ontology? It sounds like > > that would be reasonable. > > > > As for the stalled upload, a thread dump during the slow processing would > > be most helpful. On a Linux system, you can capture that by doing kill -3 > > <pid> on the Java process. It should then go to console.log. > > > > Thanks, > > David > > > > On Thu, Nov 10, 2011 at 8:21 AM, andrew ton <[email protected]> > wrote: > > > > > > > > > > > Hi David, > > > > > > Thank you for getting back to me! > > > Finally I can make it work by changing the log level from INFO (by > > > default) to FINEST in the logging.property. > > > > > > I have a question for you though. Currently my project have a problem > > with > > > uploading data to the store. I created only 1 index for the whole > > > application and the node name as the key. Several nodes in different > > > ontologies have the same names. So when an ontology is uploaded to the > > > store and has a node that its name has been already in the index (by > > > previous ontologies) this node is not created in the graph of this > > > ontology. My app can upload a number of ontologies and graphs are > created > > > successfully for each ontology in the store. However when the process > > > uploads the 9th ontology the store does not respond and it seems busy > > with > > > some internal process like looking up the node in the index or > something > > > else. I'm stuck and don't know the cause of the problem. Do you have > any > > > clue or suggestions? > > > > > > Appreciate your help! > > > > > > Regards, > > > > > > > > > ________________________________ > > > From: David Montag <[email protected]> > > > To: Peter Neubauer <[email protected]> > > > Cc: Neo4j user discussions <[email protected]> > > > Sent: Wednesday, November 9, 2011 10:07 PM > > > Subject: Re: [Neo4j] Neo4j REST server's log files > > > > > > Hi Andrew, > > > > > > Let's connect during the day tomorrow for a higher-bandwidth > discussion. > > Do > > > you have Skype? > > > > > > Thanks, > > > David > > > > > > On Wed, Nov 9, 2011 at 1:23 PM, Peter Neubauer < > > > [email protected]> wrote: > > > > > > > Andrew, > > > > this sounds like the RRD database in the server got broken. Could you > > > > delete data/rrd and start up again? Also, David is in your timezone > > > > and maybe can connect with you directly to look into this? > > > > > > > > Cheers, > > > > > > > > /peter neubauer > > > > > > > > GTalk: neubauer.peter > > > > Skype peter.neubauer > > > > Phone +46 704 106975 > > > > LinkedIn http://www.linkedin.com/in/neubauer > > > > Twitter http://twitter.com/peterneubauer > > > > > > > > http://www.neo4j.org - NOSQL for the Enterprise. > > > > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > > > > > > > > > > > > > > > > On Wed, Nov 9, 2011 at 10:07 PM, andrew ton <[email protected]> > > > wrote: > > > > > > > > > > > > > > > Hi Peter, > > > > > > > > > > I don't understand much your question. However, in my Restlet > > > > application I have a logging service using Java log to record all > > > processes > > > > in my app. My problem is that I upload 20 ontologies and after > serveral > > > > ontologies the Neo4J stops responding my REST request. It seems busy > > with > > > > some index lookup process. Consequently my app throws > > > > a NoHttpResponseException: The target server failed to respond. > > > > > I'd like to see what causes the problem inside Neo4J. Unfortunately > > > both > > > > messages.log and neo4j.x.x.log do not show run time processes. > > > > > BTW, when I start up the server the neo4j.x.x.log shows > > > > > > > > > > INFO: Server started on [http://localhost:7474/] > > > > > Nov 9, 2011 10:22:22 AM org.neo4j.server.logging.Logger log > > > > > WARNING: > > > > > java.lang.IllegalArgumentException: Bad sample time: 1320862942. > Last > > > > update time was 1320862942, at least one second step is required > > > > > at org.rrd4j.core.RrdDb.store(RrdDb.java:553) > > > > > at org.rrd4j.core.Sample.update(Sample.java:197) > > > > > at > > > > > > org.neo4j.server.rrd.RrdSamplerImpl.updateSample(RrdSamplerImpl.java:62) > > > > > at > > org.neo4j.server.rrd.RrdFactory$1.updateSample(RrdFactory.java:109) > > > > > at org.neo4j.server.rrd.RrdJob.run(RrdJob.java:43) > > > > > at org.neo4j.server.rrd.ScheduledJob$1.run(ScheduledJob.java:41) > > > > > at java.util.TimerThread.mainLoop(Timer.java:512) > > > > > at java.util.TimerThread.run(Timer.java:462) > > > > > > > > > > .... > > > > > > > > > > Nov 9, 2011 12:43:44 PM > > com.sun.jersey.api.core.PackagesResourceConfig > > > > init > > > > > INFO: Scanning for root resource and provider classes in the > > packages: > > > > > org.neo4j.server.webadmin.rest > > > > > Nov 9, 2011 12:43:44 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > logClasses > > > > > INFO: Root resource classes found: > > > > > class org.neo4j.server.webadmin.rest.MonitorService > > > > > class org.neo4j.server.webadmin.rest.RootService > > > > > class org.neo4j.server.webadmin.rest.JmxService > > > > > class org.neo4j.server.webadmin.rest.ConsoleService > > > > > Nov 9, 2011 12:43:44 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > init > > > > > INFO: No provider classes found. > > > > > Nov 9, 2011 12:43:44 PM > > > > com.sun.jersey.server.impl.application.WebApplicationImpl _initiate > > > > > INFO: Initiating Jersey application, version 'Jersey: 1.9 > 09/02/2011 > > > > 11:17 AM' > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.PackagesResourceConfig > > > > init > > > > > INFO: Scanning for root resource and provider classes in the > > packages: > > > > > org.neo4j.server.rest.web > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > logClasses > > > > > INFO: Root resource classes found: > > > > > class org.neo4j.server.rest.web.ResourcesService > > > > > class org.neo4j.server.rest.web.BatchOperationService > > > > > class org.neo4j.server.rest.web.RestfulGraphDatabase > > > > > class org.neo4j.server.rest.web.DatabaseMetadataService > > > > > class org.neo4j.server.rest.web.ExtensionService > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > logClasses > > > > > INFO: Provider classes found: > > > > > class org.neo4j.server.rest.web.DatabaseBlockedExceptionMapper > > > > > Nov 9, 2011 12:43:45 PM > > > > com.sun.jersey.server.impl.application.WebApplicationImpl _initiate > > > > > INFO: Initiating Jersey application, version 'Jersey: 1.9 > 09/02/2011 > > > > 11:17 AM' > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.PackagesResourceConfig > > > > init > > > > > INFO: Scanning for root resource and provider classes in the > > packages: > > > > > org.neo4j.server.rest.discovery > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > logClasses > > > > > INFO: Root resource classes found: > > > > > class org.neo4j.server.rest.discovery.DiscoveryService > > > > > Nov 9, 2011 12:43:45 PM > > com.sun.jersey.api.core.ScanningResourceConfig > > > > init > > > > > INFO: No provider classes found. > > > > > Nov 9, 2011 12:43:45 PM > > > > com.sun.jersey.server.impl.application.WebApplicationImpl _initiate > > > > > INFO: Initiating Jersey application, version 'Jersey: 1.9 > 09/02/2011 > > > > 11:17 AM' > > > > > > > > > > Sorry for this long mail. > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > ________________________________ > > > > > From: Peter Neubauer <[email protected]> > > > > > To: Neo4j user discussions <[email protected]> > > > > > Sent: Wednesday, November 9, 2011 10:29 AM > > > > > Subject: Re: [Neo4j] Neo4j REST server's log files > > > > > > > > > > Andrew, > > > > > Go you have the generating code somewhere you check and reproduce? > > > > > On Nov 9, 2011 6:26 PM, "andrew ton" <[email protected]> wrote: > > > > > > > > > >> > > > > >> > > > > >> Hi Peter, > > > > >> > > > > >> I tried the messages.log but it only showed processes up to the > time > > > > when > > > > >> the server is up > > > > >> > > > > >> 2011-11-09 08:59:19.669-0800: --- CONFIGURATION END --- > > > > >> 2011-11-09 08:59:19.733-0800: Extension > > > > >> org.neo4j.kernel.KernelExtension[kernel jmx] loaded ok > > > > >> 2011-11-09 08:59:19.857-0800: Extension > > > > >> org.neo4j.kernel.KernelExtension[shell] loaded ok > > > > >> 2011-11-09 08:59:19.858-0800: Extension > > > > >> org.neo4j.kernel.KernelExtension[kernel udc] loaded ok > > > > >> > > > > >> The manual shows that the logging is configured in > > > > conf/logging.properties > > > > >> (java.util.logging.FileHandler.pattern=data/log/neo4j.%u.%g.log). > > I'm > > > > using > > > > >> the default settings but the log file was not updated in the run > > time. > > > > >> My purpose is that I want to see what is wrong when I failed to > > store > > > an > > > > >> ontology into Neo4J. > > > > >> Please help me out! > > > > >> > > > > >> Thanks and regards, > > > > >> > > > > >> > > > > >> > > > > >> ________________________________ > > > > >> From: Peter Neubauer <[email protected]> > > > > >> To: Neo4j user discussions <[email protected]> > > > > >> Sent: Wednesday, November 9, 2011 8:46 AM > > > > >> Subject: Re: [Neo4j] Neo4j REST server's log files > > > > >> > > > > >> Andrew, > > > > >> The database logs are in data/graphdb / messages.log > > > > >> On Nov 9, 2011 5:39 PM, "andrew ton" <[email protected]> > wrote: > > > > >> > > > > >> > > > > > >> > > > > > >> > Hi, > > > > >> > > > > > >> > What log files of the Neo4J REST server can I use to check > what's > > > > going > > > > >> on > > > > >> > in the server? The log/console.log and neo4j.x.x.log do not show > > > > >> > transactions in the run time. I also looked into > > > graph.db/tm_tx_log.x > > > > >> but I > > > > >> > did not learn anything from there. Can somebody give me some > > > pointers? > > > > >> > > > > > >> > Thanks, > > > > >> > _______________________________________________ > > > > >> > Neo4j mailing list > > > > >> > [email protected] > > > > >> > https://lists.neo4j.org/mailman/listinfo/user > > > > >> > > > > > >> _______________________________________________ > > > > >> Neo4j mailing list > > > > >> [email protected] > > > > >> https://lists.neo4j.org/mailman/listinfo/user > > > > >> _______________________________________________ > > > > >> Neo4j mailing list > > > > >> [email protected] > > > > >> https://lists.neo4j.org/mailman/listinfo/user > > > > >> > > > > > _______________________________________________ > > > > > Neo4j mailing list > > > > > [email protected] > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > _______________________________________________ > > > > > Neo4j mailing list > > > > > [email protected] > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > > > > > > > > > > -- > > > David Montag <[email protected]> > > > Neo Technology, www.neotechnology.com > > > Cell: 650.556.4411 > > > Skype: ddmontag > > > _______________________________________________ > > > Neo4j mailing list > > > [email protected] > > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > > > Neo4j mailing list > > > [email protected] > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > -- > > David Montag <[email protected]> > > Neo Technology, www.neotechnology.com > > Cell: 650.556.4411 > > Skype: ddmontag > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > -- > David Montag <[email protected]> > Neo Technology, www.neotechnology.com > Cell: 650.556.4411 > Skype: ddmontag > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- David Montag <[email protected]> Neo Technology, www.neotechnology.com Cell: 650.556.4411 Skype: ddmontag _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
