I'm having another stab at trying to resolve this. I believe that the
following two logs are strongly connected:
2015-02-23 13:35:25 ERROR failed to parse json string, lexical error:
invalid char in json text.; A
connection attempt failed bec; (right here) ------^; [A
connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond. ]
2015-02-23 13:35:25 ERROR procedure 'parse_json' failed at line 123,
character 25 in /etc/nxlog.conf. statement execution has been aborted;map
key name not found
The text "A connection attempt failed because ..." is something that the
WinSock (or something that depends on it) emits as a standard error. I
think its probably occurring in one or more of my client nxlog senders;
perhaps likely due to the frequent restarts my server nxlog experiences
(due to memory issues and resultant hanging / restart by watchdog). So
somewhere (possibly in nxlog) an error message is being injected into the
protocol stream. The first message (or rather, the text quoted in the
message) is from Windows (ie. nxlog sender), and the second is a resultant
(most likely) error reported on Linux (ie. nxlog receiver).
I've created a patch for json.c which (if available) lists the client IP
address in the log:
$ diff -N ./src/modules/extension/json/json.c{.pre-crk,}
444,445c444,455
<
< log_error("failed to parse json string, %s [%s]", errstr, json);
---
> nx_logdata_field_t *maybe_msa;
>
> maybe_msa = nx_logdata_get_field(ctx->logdata,
"MessageSourceAddress");
>
> if (maybe_msa != NULL && maybe_msa->value->type ==
NX_VALUE_TYPE_STRING && maybe_msa->value->defined)
> {
> log_error("[client %s] failed to parse json string, %s [%s]",
maybe_msa->value->string->buf, errstr, json);
> }
> else
> {
> log_error("failed to parse json string, %s [%s]", errstr,
json);
> }
This patch only modifies nx_json_parse()
I've deployed this to my production environment and it is working (ie.
identifying the clients sending such rubbish).
As to why a client may be sending rubbish in the first place, I suspect my
sending clients are not always running 'to_json()'.
Cheers,
Cameron
--
Cameron Kerr <cameron.kerr...@gmail.com>
See my blog at http://distracted-it.blogspot.co.nz/ (previously
http://humbledown.org/)
Skype me on cameron.kerr.nz
On 26 August 2014 at 14:31, Cameron Kerr <cameron.kerr...@gmail.com> wrote:
> Hi all,
>
> I'm trying to track down an error (one of a few that is prevalent in my
> logs on one of my central NXLOG instances).
>
> ERROR procedure 'parse_json' failed at line 110, character 25 in
> /etc/nxlog.conf. *statement execution has been aborted;map key name not
> found*
>
>
> Here it is as part of a sequence of logs (whether there are relationships
> between this entries, I don't know; it is a busy log server).
>
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid string in json text.;
> function 'replace' failed at li; (right here) ------^;
> [function 'replace' failed at line 104, character 152 in C:\Program Files
> (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted]
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid char in json text.; 'unknown'
> type argument is inva; (right here) ------^; ['unknown'
> type argument is invalid for argument 'subject', string required]
> 2014-08-26 12:24:55 ERROR procedure 'parse_json' failed at line 110,
> character 25 in /etc/nxlog.conf. statement execution has been aborted;map
> key name not found
> 2014-08-26 12:24:55 ERROR procedure 'parse_json' failed at line 110,
> character 25 in /etc/nxlog.conf. statement execution has been aborted;map
> key name not found
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid string in json text.;
> function 'replace' failed at li; (right here) ------^;
> [function 'replace' failed at line 104, character 152 in C:\Program Files
> (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted]
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid char in json text.; 'unknown'
> type argument is inva; (right here) ------^; ['unknown'
> type argument is invalid for argument 'subject', string required]
> 2014-08-26 12:24:55 ERROR procedure 'parse_json' failed at line 110,
> character 25 in /etc/nxlog.conf. statement execution has been aborted;map
> key name not found
> 2014-08-26 12:24:55 ERROR procedure 'parse_json' failed at line 110,
> character 25 in /etc/nxlog.conf. statement execution has been aborted;map
> key name not found
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid string in json text.;
> function 'replace' failed at li; (right here) ------^;
> [function 'replace' failed at line 104, character 152 in C:\Program Files
> (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted]
> 2014-08-26 12:24:55 ERROR failed to parse json string, lexical error:
> invalid char in json text.; 'unknown'
> type argument is inva; (right here) ------^; ['unknown'
> type argument is invalid for argument 'subject', string required]
> 2014-08-26 12:24:55 ERROR procedure 'parse_json' failed at line 110,
> character 25 in /etc/nxlog.conf. statement execution has been aborted;map
> key name not found
>
>
> I think that's talking about this particular invocation (line 110 is the
> last line of this Exec statement.
>
> <Input in_json_tls>
> Module im_ssl
> Port 10514
> Host 0.0.0.0
> CertKeyFile /etc/pki/tls/private/logs.MYSITE.key
> CertFile /etc/pki/tls/certs/logs.MYSITE.crt
> CAFile /etc/pki/tls/certs/logs.MYSITE.ca-bundle
> RequireCert FALSE
> Exec parse_json(); \
> if $MY_application_stack == undef \
> { \
> $MY_application_stack = "common"; \
> } \
> if $MY_log_type == undef \
> { \
> $MY_log_type = "unknown"; \
> } \
> if $MY_environment == undef \
> { \
> $MY_environment = "prod"; \
> }
> </Input>
>
>
> I would like some insight into what the "map key name not found" means, so
> I can look for what logs might be causing that.
>
> I see in the source code that "map key name not found" is always
> associated with a handful of call-sites in the JSON module, and always like
> the following:
>
> if ( ctx->key == NULL )
> {
> throw_msg("map key name not found");
> }
>
> In particular, I see that it is potentially called inside the the
> callbacks for all of the various JSON datatypes (ie. in yajl_parse_*_cb).
> Looking in the code, it seems that this sort of thing is never expected,
> and is perhaps symptomatic of a bug.
>
>
> It would be better if throw_msg included __FILE__, __LINE__ and
> __FUNCTION__, then I would know exactly which call-site to look at.
>
>
> Cheers,
> Cameron
>
> --
> Cameron Kerr <cameron.kerr...@gmail.com>
> See my blog at http://distracted-it.blogspot.co.nz/ (previously
> http://humbledown.org/)
> Skype me on cameron.kerr.nz
>
>
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
