Decided to run nxlog solo with the correct information and still get the
same problem:
2016-06-23T12:58:18.248-04:00 ERROR [GelfCodec] Could not parse JSON, first
400 characters: `�)�V���C�
�a?�n
�n�r��埯o����}ۍvdY>_"~g��rgИ:�
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('`'
(code 96)): expected a valid value (number, String, array, object, 'true',
'false' or 'null')
at [Source: `�)�V���C�
�a?�n
�n�r��埯o����}ۍvdY>_"~g��rgИ:�; line: 1, column: 2]
at
com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1624)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:689)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3771)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3716)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2272)
~[graylog.jar:?]
at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:115)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:136)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:82)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:58)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
[graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139)
[graylog.jar:?]
at
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
2016-06-23T12:58:18.253-04:00 ERROR [DecodingProcessor] Unable to decode
raw message af0450e2-3963-11e6-ae65-005056937893 (journal offset 84466333)
encoded as gelf received from /10.100.150.89:34338.
Then took HAProxy out of the picture and pushed to one of the servers
directly and STILL get the same problem:
2016-06-23T13:01:41.359-04:00 ERROR [GelfCodec] Could not parse JSON, first
400 characters: �p]3
b�Q��F!��0�и
i�D��
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('�'
(code 65533 / 0xfffd)): expected a valid value (number, String, array,
object, 'true', 'false' or 'null')
at [Source: �p]3
b�Q��F!��0�и
i�D��; line: 1, column: 2]
at
com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1624)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:689)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3771)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3716)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2272)
~[graylog.jar:?]
at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:115)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:136)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:82)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:58)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
[graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139)
[graylog.jar:?]
at
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
2016-06-23T13:01:41.359-04:00 ERROR [DecodingProcessor] Unable to decode
raw message 2808ffa4-3964-11e6-ae65-005056937893 (journal offset 84512788)
encoded as gelf received from /10.100.1.10:50911.
Last, ran it again a server built from scratch with graylog-server-2.0.3-1
and still have the same issue.
2016-06-23T13:09:01.840-04:00 ERROR [GelfCodec] Could not parse JSON, first
400 characters: �$��D�Q�ݑ�/[�m��.
umv�6Ӭ�����5>btt��h�Rx��S]o�0�+V�Z�AvBȞ���
�����y��)歷�1ń��x�����W-�\�{0V�&��a�]��Nm�
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('�'
(code 65533 / 0xfffd)): expected a valid value (number, String, array,
object, 'true', 'false' or 'null')
at [Source: �$��D�Q�ݑ�/[�m��. umv�6Ӭ�����5>btt��h�Rx��S]o�0�+V�Z�AvBȞ���
�����y��)歷�1ń��x�����W-; line: 1, column: 2]
at
com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1624)
~[graylog.jar:?]
at
com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:689)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3771)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3716)
~[graylog.jar:?]
at
com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2272)
~[graylog.jar:?]
at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:115)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:136)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:82)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:58)
[graylog.jar:?]
at
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
[graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139)
[graylog.jar:?]
at
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
2016-06-23T13:09:01.857-04:00 ERROR [DecodingProcessor] Unable to decode
raw message 2e97f704-3965-11e6-b0e7-005056930c85 (journal offset 78064)
encoded as gelf received from /10.100.1.10:61360.
A problem with NXLog or Graylog 2.0.3-1? This issue did not exist before
the upgrade and the only change is the upgrade. Not sure what is happening
here?
Thanks - Shon
On Thursday, June 23, 2016 at 11:08:39 AM UTC-4, Shon Nixon wrote:
> All my NXlog files look like:
>
>
> define ROOT C:\Program Files (x86)\nxlog
>
> <Extension gelf>
>
> Module xm_gelf
>
> </Extension>
>
> define ROOT C:\Program Files (x86)\nxlog
>
> Moduledir %ROOT%\modules
>
> CacheDir %ROOT%\data
>
> Pidfile %ROOT%\data\nxlog.pid
>
> SpoolDir %ROOT%\data
>
> LogFile %ROOT%\data\nxlog.log
>
> <Extension gelf>
>
> Module xm_gelf
>
> </Extension>
>
> <Input in>
>
> Module im_msvistalog
>
> </Input>
>
> <Output out>
>
> Module om_tcp
>
> Host <host ip address>
>
> Port 12201
>
> OutputType GELF
>
> </Output>
>
> <Route r>
>
> Path in => out
>
> </Route>
>
>
>
> All I did was perform the upgrade of Graylog, nothing more. I always check
> log files to make sure the new upgrade took and that's when this started
> happening--immediately.
>
>
>
>
> Thanks - Shon
>
>
>
> On Thursday, June 23, 2016 at 11:05:56 AM UTC-4, Shon Nixon wrote:
>
>> It would appear that Graylog is adding additional lines in the NXlog
>> file. My snippet is:
>>
>> define ROOT C:\Program Files (x86)\nxlog
>> Moduledir %ROOT%\modules
>> CacheDir %ROOT%\data
>> Pidfile %ROOT%\data\nxlog.pid
>> SpoolDir %ROOT%\data
>> LogFile %ROOT%\data\nxlog.log
>>
>> <Extension gelf>
>> Module xm_gelf
>> </Extension>
>>
>> <Input in>
>> Module im_msvistalog
>> </Input>
>>
>> <Output out>
>> Module om_tcp
>> Host 10.100.150.89
>> Port 12201
>> OutputType GELF
>> </Output>
>>
>> <Route r>
>> Path in => out
>> </Route>
>>
>>
>> Graylog is adding:
>>
>> define ROOT C:\Program Files (x86)\nxlog
>>
>> <Extension gelf>
>>
>> Module xm_gelf
>>
>> </Extension>
>>
>>
>> to the top of every Nxlog file.
>>
>>
>> Many Thanks - Shon
>>
>>
>>
>> On Thursday, June 23, 2016 at 6:05:55 AM UTC-4, Marius Sturm wrote:
>>
>>> Hi,
>>> looks like your receiving some binary data on a plain text Gelf input.
>>> Did you switch to TLS encryption or soemthing like that after the update?
>>> Could you please post the generated configuration of NXlog?
>>>
>>> Cheers,
>>> Marius
>>>
>>> On Wednesday, 22 June 2016 16:27:41 UTC+2, Shon Nixon wrote:
>>>>
>>>> Built a Graylog 2.0 cluster two weeks ago with three servers running
>>>> 2.0.2 behind a HAProxy server. Server accepts logs from all Windows boxes
>>>> using Graylog Sidecar and Nxlog. Was working perfectly until I upgraded
>>>> the
>>>> server to 2.0.3 (yum upgrade). Restarted the services and now I get a
>>>> constant flow of the log info below. Also can no longer access
>>>> *System/Inputs
>>>> -> Configurations* page. Just shows blank. I do have another two box
>>>> cluster I built yesterday that started with 2.0.3 and has no issues at all
>>>> getting data from the HAProxy box. I would rather not rebuild this cluster
>>>> if possible...
>>>>
>>>>
>>>> Reoccurring logging on all three boxes:
>>>>
>>>>
>>>> 2016-06-22T10:15:13.732-04:00 ERROR [GelfCodec] Could not parse JSON,
>>>> first 400 characters: ���vb� ����wxz�Tv��<�Q���u]?�I� �z��
>>>> com.fasterxml.jackson.core.JsonParseException: Unexpected character
>>>> ('�' (code 65533 / 0xfffd)): expected a valid value (number, String,
>>>> array,
>>>> object, 'true', 'false' or 'null')
>>>> at [Source: ���vb� ����wxz�Tv��<�Q���u]?�I� �z��; line: 1, column:
>>>> 2]
>>>> at
>>>> com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1624)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:689)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3771)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3716)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2272)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:115)
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:136)
>>>>
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:82)
>>>>
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:58)
>>>>
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
>>>>
>>>> [graylog.jar:?]
>>>> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139)
>>>> [graylog.jar:?]
>>>> at
>>>> com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
>>>>
>>>> [graylog.jar:?]
>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
>>>> 2016-06-22T10:15:13.732-04:00 ERROR [DecodingProcessor] Unable to
>>>> decode raw message bc89d68a-3883-11e6-a89e-005056934db8 (journal offset
>>>> 56420554) encoded as gelf received from /10.100.150.89:41076.
>>>> 2016-06-22T10:15:13.732-04:00 ERROR [DecodingProcessor] Error
>>>> processing message RawMessage{id=bc89d68a-3883-11e6-a89e-005056934db8,
>>>> journalOffset=56420554, codec=gelf, payloadSize=41,
>>>> timestamp=2016-06-22T14:15:13.640Z, remoteAddress=/10.100.150.89:41076}
>>>> com.fasterxml.jackson.core.JsonParseException: Unexpected character
>>>> ('�' (code 65533 / 0xfffd)): expected a valid value (number, String,
>>>> array,
>>>> object, 'true', 'false' or 'null')
>>>> at [Source: ���vb� ����wxz�Tv��<�Q���u]?�I� �z��; line: 1, column:
>>>> 2]
>>>> at
>>>> com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1624)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:689)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3771)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3716)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2272)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:115)
>>>> ~[graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:136)
>>>>
>>>> ~[graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:82)
>>>>
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:58)
>>>>
>>>> [graylog.jar:?]
>>>> at
>>>> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
>>>>
>>>> [graylog.jar:?]
>>>> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139)
>>>> [graylog.jar:?]
>>>> at
>>>> com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
>>>>
>>>> [graylog.jar:?]
>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/25b2ce8f-58f5-4d10-8cae-32caa384ad40%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
