I think I figured out what was going on. One of the code paths for graph query was missing a finish() call. I think this was causing a lot of transactions to 'work' but then eventually timeout and to dump to the messages.log file.
Still verifying that this is the issue but right now disk growth looks a lot better. Adam On Tuesday, 15 July 2014 10:01:59 UTC-7, Adam Lofts wrote: > > Hi Michael, > > Truncating it regularly would work and I could write a cron job to do it. > Right now I'm looking for a more robust solution since in some sense these > folders are managed by the neo4j and it seems dangerous to interfere with > them from another process. An integrated solution would still work if the > daemon gets re-deployed into a different folder for example. > > Example messages.log file (there are lots of these on the server). Note it > is 1.5 GB in size. > > -rw-r--r-- 1 root root 1.5G Jul 15 16:33 > /var/www/carboncloud/couchdb-neo4j-deft/couchdb-neo4j-deft-0.9.1/indexes/89d4d8e6-e072-44a6-9852-6598013560e3/messages.log > > Example content from the start of the file: > > 2014-03-19 15:28:07,477 INFO [neo4j.txmanager]: TM new log: tm_tx_log.1 >> 2014-03-19 15:28:07,483 INFO [neo4j.xafactory]: Opened logical log >> [/var/www/carboncloud/couchdb-neo4j-deft/couchdb-neo4j-deft-0.9.1/indexes/89d4d8e6-e072-44a6-9852-6598013560e3/index/lucene.log.1] >> >> version=0, lastTxId=1 (clean) >> 2014-03-19 15:28:07,488 DEBUG [neo4j.diagnostics]: --- STARTUP >> diagnostics START --- >> 2014-03-19 15:28:07,488 DEBUG [neo4j.diagnostics]: Neo4j Kernel >> properties: >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: forced_kernel_id= >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: read_only=false >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: neo4j.ext.udc.host= >> udc.neo4j.org >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: >> logical_log=nioneo_logical.log >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: node_aut >> o_indexing=true >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: >> intercept_committing_transactions=false >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: cache_type=soft >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: >> intercept_deserialized_transactions=false >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: >> lucene_searcher_cache_size=2147483647 >> 2014-03-19 15:28:07,489 DEBUG [neo4j.diagnostics]: >> neo4j.ext.udc.interval=86400000 >> > > The problem may be that I open and close the index a lot. Maybe I can > reduce the debug output when doing this? > > Thanks! > > On Monday, 14 July 2014 17:11:58 UTC-7, Michael Hunger wrote: >> >> Does it help to truncate it regularly? >> >> What is the actual memory issue? Perhaps you can share your messages.log >> file? >> >> Michael >> >> Am 14.07.2014 um 00:11 schrieb Adam Lofts <[email protected]>: >> >> Hi, >> >> I am running some servers which open lots of neo4j indexes. These servers >> are 'on the limit' of memory capacity so there is a log of logging output >> to messages.log. Is there some way I can limit the size of the messages.log >> file or alternatively turn off all logging to this file? The disk usage >> just from all the messages.log files is a problem. >> >> Thanks! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
