Hey Tristan Yes we did restart graylog2-server. In fact we stopped it, deleted the files in /var/lib/graylog2-server/message-cache-spool/* and then started it again. Still getting these exceptions..
On Wednesday, December 10, 2014 6:10:45 PM UTC+1, Tristan Rhodes wrote: > > Sandro, > > We had these same errors on version 0.92, but after reading the release > notes for 0.92.1 I disabled the output buffer and then restarted > graylog2. > > disable_output_cache = true > > Now I don't get the messages anymore! > > Did you restart the graylog2 service after disabling the output cache? > > sudo service graylog2-server restart > > Cheers! > > Tristan > > > > On Wed, Dec 10, 2014 at 6:06 AM, Sandro Roth <[email protected] > <javascript:>> wrote: > >> We are seeing the same issue. Deleted buffer files and disabled output >> buffer, but that didn't help.. >> Let's hope they fixed this in 0.92.1 >> >> >> On Monday, December 8, 2014 2:12:10 PM UTC+1, Razvan Grigore wrote: >>> >>> We have the same problem, after updating to 0.92, some at startup: >>> >>> 2014-12-08T13:54:55.590+01:00 ERROR [MasterCacheWorkerThread] Error >>> while trying to work on MasterCache <org.graylog2.caches. >>> DiskJournalCache.Output>. >>> java.lang.NullPointerException >>> at org.mapdb.Queues$SimpleQueue.peek(Queues.java:117) >>> at org.mapdb.Queues$SimpleQueue.isEmpty(Queues.java:287) >>> at org.graylog2.caches.DiskJournalCache.isEmpty( >>> DiskJournalCache.java:238) >>> at org.graylog2.periodical.MasterCacheWorkerThread.work( >>> MasterCacheWorkerThread.java:106) >>> at org.graylog2.periodical.MasterCacheWorkerThread.access$200( >>> MasterCacheWorkerThread.java:43) >>> at org.graylog2.periodical.MasterCacheWorkerThread$2.run( >>> MasterCacheWorkerThread.java:87) >>> at java.lang.Thread.run(Thread.java:745) >>> 2014-12-08T13:54:57.101+01:00 ERROR [AbstractCacheWorkerThread] >>> Exception while working on cache: >>> java.lang.NullPointerException >>> at org.mapdb.Queues$SimpleQueue.poll(Queues.java:135) >>> at org.mapdb.Queues$SimpleQueue.drainTo(Queues.java:271) >>> at org.graylog2.caches.DiskJournalCache.drainTo( >>> DiskJournalCache.java:207) >>> at org.graylog2.periodical.AbstractCacheWorkerThread. >>> drainMessagesEnqueue(AbstractCacheWorkerThread.java:79) >>> at org.graylog2.periodical.AbstractCacheWorkerThread.work( >>> AbstractCacheWorkerThread.java:58) >>> at org.graylog2.periodical.OutputCacheWorkerThread$1.run( >>> OutputCacheWorkerThread.java:60) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> ... but also a shorter one all the time, each few seconds: >>> >>> >>> 2014-12-08T13:56:47.912+01:00 ERROR [MasterCacheWorkerThread] Error >>> while trying to work on MasterCache <org.graylog2.caches. >>> DiskJournalCache.Output>. >>> java.lang.NullPointerException >>> 2014-12-08T13:56:57.332+01:00 ERROR [MasterCacheWorkerThread] Error >>> while trying to work on MasterCache <org.graylog2.caches. >>> DiskJournalCache.Output>. >>> java.lang.NullPointerException >>> 2014-12-08T13:57:17.003+01:00 ERROR [AbstractCacheWorkerThread] >>> Exception while working on cache: >>> java.lang.NullPointerException >>> >>> I already tried to delete the contents of that folder and restart the >>> graylog-server, but still the same. >>> >>> Razvan >>> >>> >>> On Tuesday, December 2, 2014 11:44:53 AM UTC+1, Jochen Schalanda wrote: >>>> >>>> Hi Martin, >>>> >>>> please try deleting the files in your message_cache_spool_dir (see >>>> graylog2.conf). >>>> >>>> >>>> Cheers, >>>> Jochen >>>> >>>> On Tuesday, 2 December 2014 10:16:38 UTC+1, Martin René Mortensen wrote: >>>>> >>>>> After upgrading to 0.92 Im having troubles with >>>>> java.lang.NullPointerException in GL2 server logs. I have 3 graylog2 >>>>> servers all reporting the same errors periodically. >>>>> >>>>> I'm getting these when starting: >>>>> >>>>> 2014-12-01T14:09:33.416+01:00 INFO [node] [node2] initialized >>>>> 2014-12-01T14:09:33.685+01:00 INFO [DiskJournalCache] Compacting >>>>> off-heap message cache database files (output-cache) >>>>> 2014-12-01T14:09:36.913+01:00 INFO [OutputBuffer] Initialized >>>>> OutputBuffer with ring size <8192> and wait strategy >>>>> <BlockingWaitStrategy>. >>>>> 2014-12-01T14:09:36.982+01:00 INFO [Main] Graylog2 Server 0.92.0 >>>>> (c1fe236) starting up. (JRE: Oracle Corporation 1.7.0_72 on Linux >>>>> 3.10.0-123.9.3.el7.x86_64) >>>>> 2014-12-01T14:09:37.034+01:00 INFO [PeriodicalsService] Starting 17 >>>>> periodicals ... >>>>> 2014-12-01T14:09:37.038+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.TelemetryReporterThread] periodical. Not >>>>> configured to run on this node. >>>>> 2014-12-01T14:09:37.038+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.StreamThroughputCounterManagerThread] >>>>> periodical in [0s], polling every [1s]. >>>>> 2014-12-01T14:09:37.039+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.InputCacheWorkerThread] periodical, running >>>>> forever. >>>>> 2014-12-01T14:09:37.039+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.IndexerClusterCheckerThread] periodical. >>>>> Only started on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.039+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.ClusterHealthCheckThread] periodical. Only >>>>> started on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.039+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. >>>>> Only started on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.039+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.AlertScannerThread] periodical. Only started >>>>> on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.040+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] >>>>> periodical in [0s], polling every [5s]. >>>>> 2014-12-01T14:09:37.040+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, >>>>> running forever. >>>>> 2014-12-01T14:09:37.041+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.IndexRotationThread] periodical. Only >>>>> started on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.041+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.ThroughputCounterManagerThread] periodical >>>>> in [0s], polling every [1s]. >>>>> 2014-12-01T14:09:37.042+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.MasterCacheWorkerThread] periodical, running >>>>> forever. >>>>> 2014-12-01T14:09:37.043+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.DeadLetterThread] periodical, running >>>>> forever. >>>>> 2014-12-01T14:09:37.044+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.VersionCheckThread] periodical. Only started >>>>> on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.044+01:00 INFO [PeriodicalsService] Not starting >>>>> [org.graylog2.periodical.IndexRetentionThread] periodical. Only >>>>> started on graylog2-server master nodes. >>>>> 2014-12-01T14:09:37.044+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.OutputCacheWorkerThread] periodical, running >>>>> forever. >>>>> 2014-12-01T14:09:37.044+01:00 INFO [Periodicals] Starting >>>>> [org.graylog2.periodical.NodePingThread] periodical in [0s], polling >>>>> every [1s]. >>>>> 2014-12-01T14:09:37.050+01:00 INFO [node] [node2] starting ... >>>>> 2014-12-01T14:09:37.270+01:00 INFO [transport] [node2] bound_address >>>>> {inet[/0:0:0:0:0:0:0:0:9350]}, publish_address {inet[/1.1.1.107:9350]} >>>>> 2014-12-01T14:09:37.279+01:00 ERROR [MasterCacheWorkerThread] Error >>>>> while trying to work on MasterCache <org.graylog2.caches. >>>>> DiskJournalCache.Output>. >>>>> java.lang.NullPointerException >>>>> at org.mapdb.Queues$SimpleQueue.peek(Queues.java:117) >>>>> at org.mapdb.Queues$SimpleQueue.isEmpty(Queues.java:287) >>>>> at org.graylog2.caches.DiskJournalCache.isEmpty( >>>>> DiskJournalCache.java:238) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread.work( >>>>> MasterCacheWorkerThread.java:102) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread.access$200( >>>>> MasterCacheWorkerThread.java:43) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread$2.run( >>>>> MasterCacheWorkerThread.java:87) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> 2014-12-01T14:09:37.305+01:00 INFO [discovery] [node2] >>>>> spocklog/zvu6TQifTVe0eBV_h9NU0A >>>>> 2014-12-01T14:09:38.497+01:00 ERROR [MasterCacheWorkerThread] Error >>>>> while trying to work on MasterCache <org.graylog2.caches. >>>>> DiskJournalCache.Output>. >>>>> java.lang.NullPointerException >>>>> at org.mapdb.Queues$SimpleQueue.peek(Queues.java:117) >>>>> at org.mapdb.Queues$SimpleQueue.isEmpty(Queues.java:287) >>>>> at org.graylog2.caches.DiskJournalCache.isEmpty( >>>>> DiskJournalCache.java:238) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread.work( >>>>> MasterCacheWorkerThread.java:102) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread.access$200( >>>>> MasterCacheWorkerThread.java:43) >>>>> at org.graylog2.periodical.MasterCacheWorkerThread$2.run( >>>>> MasterCacheWorkerThread.java:87) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> 2014-12-01T14:09:38.763+01:00 INFO [KieRepositoryImpl] KieModule was >>>>> added:MemoryKieModule[ ReleaseId=org.graylog2:dynamic-rules:0] >>>>> 2014-12-01T14:09:38.852+01:00 INFO [KieRepositoryImpl] Adding >>>>> KieModule from resource :[ByteArrayResource resource=[B@5da78d54] >>>>> 2014-12-01T14:09:38.961+01:00 INFO [KieRepositoryImpl] KieModule was >>>>> added:MemoryKieModule[ ReleaseId=org.graylog2:dynamic-rules:0] >>>>> 2014-12-01T14:09:39.475+01:00 WARN [BatchedElasticSearchOutput] >>>>> Clearing buffer (5000 messages) because the Elasticsearch cluster is down. >>>>> 2014-12-01T14:09:40.179+01:00 WARN [BatchedElasticSearchOutput] >>>>> Clearing buffer (5000 messages) because the Elasticsearch cluster is down. >>>>> 2014-12-01T14:09:40.313+01:00 WARN [discovery] [node2] waited for 3s >>>>> and no initial state was set by the discovery >>>>> 2014-12-01T14:09:40.315+01:00 INFO [node] [node2] started >>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "graylog2" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Tristan Rhodes > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
