I meant to weigh in on this the other day...
Yes, VMware logs are not the best to work with between multiline formats,
facilities that are random at best, and so forth.
Before implementing Graylog2, I set up a centralised syslog server with
CentOS 7 and the provided rsyslogd to capture all our vSphere logs (5.5).
Even when implementing Graylog2 we needed to retain the centralised syslog
server as we run ESXi embedded and this was the only way we could retain a
copy of the logs in a format that we could tar up and send to VMware
Support if necessary.
All I did from here was configure rsyslogd to forward the logs to the
Graylog2 Syslog input in RFC 5424 format and it seems to work acceptably
well.
The only extractors I have created so far are to extract some SCSI command
and error information out when SCSI errors occur. I will post the
extractors below.
I also have vCenter logs being sent in via NXLog to a GELF UDP input and
while I haven't done anything magical with these in terms of extractors,
most things seem to be searchable given the few scenarios I've needed to
look into them.
Here's the extractors I have to get things like the device NAA ID, SCSI
command, Sense data, and so forth:
{
"extractors": [
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice.*",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP:.*Cmd (0x..).*"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "Cmd",
"title": "ESXi: Extract SCSI Command"
},
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice:.*",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP: nmp_ThrottleLogForDevice:.*dev
\"(.*?)\".*"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "Device",
"title": "ESXi: Extract Device"
},
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice:.*",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP: nmp_ThrottleLogForDevice:.*path
\"(.*?)\".*"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "Path",
"title": "ESXi: Extract Path"
},
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice.*",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP: nmp_ThrottleLogForDevice.*Failed:
(H:0x.{1,2} D:0x.{1,2} P:0x.{1,2}).*"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "Status",
"title": "ESXi: Extract Host, Device, or NMP Plugin Status"
},
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP: nmp_ThrottleLogForDevice.*sense data:
(0x.{1,2} 0x.{1,2} 0x.{1,2}).*"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "SenseData",
"title": "ESXi: Extract Sense Data"
},
{
"condition_type": "regex",
"condition_value": "(?i).*NMP: nmp_ThrottleLogForDevice",
"converters": [],
"cursor_strategy": "copy",
"extractor_config": {
"regex_value": "(?i).*NMP: nmp_ThrottleLogForDevice.*Act:(.*)$"
},
"extractor_type": "regex",
"order": 0,
"source_field": "message",
"target_field": "Action",
"title": "ESXi: Extract pathing action"
}
],
"version": "0.92.3"
}
On Wednesday, 11 March 2015 10:19:35 UTC+10, Peter Loron wrote:
>
> Hmm. So I may need to parse it through NXlog or somesuch first. Bummer.
>
> On Friday, March 6, 2015 at 10:23:10 AM UTC-8, Henrik Johansen wrote:
>>
>> The VMware log format is a gigantic PITA - AFAIK graylog cannot handle
>> multiline syslog at this time ...
>>
>>
>>
>> ---
>> Sent from a mobile device
>>
>> On 6. mar. 2015 kl. 19.09.25 CET, Peter Loron <[email protected]> wrote:
>>
>> Hi. We're running Graylog 1.0, and have a couple of VMware ESXi 5.5
>> clusters. Sadly, the messages coming from ESXi don't conform to the syslog
>> standards. Anybody have a working extractor for these? Thanks.
>>
>> -Pete
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "graylog2" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
