Re: [rsyslog] Malfored syslog records missing host and extra date

[David Lang]

you didn't include the full debug message for the bad case (missing rawmsg, 
which shows what it receives)


On Mon, 21 Aug 2017, Don M Subscriptions wrote:


Date: Mon, 21 Aug 2017 20:12:19 -0400
From: Don M Subscriptions <donmrdch.subscripti...@gmail.com>
To: David Lang <da...@lang.hm>, rsyslog@lists.adiscon.com
Subject: Re: [rsyslog] Malfored syslog records missing host and extra date

First, thanks for the input!

There are two lines posted below. The first one is from the local system, its 
own IPtables record. To me, this sure looks right. The second one is the line 
from the external system, which is the syslog line that is incorrect. I did 
query the vendor, and it is thier direction that I "figure out how to get rid 
of or strip off my own rsyslog date". From what I can glean, this means that 
I need to create a "template" and then apply that specific template to data 
arriving from thier host. I did change a the data to sanitize it a bit. My 
changes are in UPPERCASE to make that clearer (I hope).


I think I need something like this:

# template to improve data from vendor cloud (Aug 21, 2017, DJM)
$template customFormat,"%HOSTNAME% %syslogtag%%msg%\n"
# $ActionFileDefaultTemplate customFormat
 RULES 
if $fromhost == 'ec2-VNDR.INET-ADDR.HERE.compute-1.amazonaws.com' then 
-/var/log/data-from-vendor;customFormat
if $fromhost-ip == 'VNDR.INET-ADDR.HERE' then 
-/var/log/data-from-vendor;customFormat



Debug line with all properties: NORMAL line.
FROMHOST: 'LOCALHOST_NAME_HERE', fromhost-ip: '127.0.0.1', HOSTNAME: 
'LOCALHOST_NAME_HERE', PRI: 6,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '', 
MSGID: '-',

TIMESTAMP: 'Aug 19 21:28:51', STRUCTURED-DATA: '-',
msg: 'iptables: IN=eth0 OUT= MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 
SRC=VNDR.INET-ADDR.HERE DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 
TTL=52 ID=2154 DF PROTO=TCP SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN 
URGP=0 '
escaped msg: 'iptables: IN=eth0 OUT= 
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=VNDR.INET-ADDR.HERE 
DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=2154 DF PROTO=TCP 
SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0 '
inputname: imklog rawmsg: 'iptables: IN=eth0 OUT= 
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=VNDR.INET-ADDR.HERE 
DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=2154 DF PROTO=TCP 
SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0 '


Debug line with all properties: SUSPECT-IMPROPER-UNHAPPY line
FROMHOST: 'ec2-VNDR.INET-ADDR.HERE.compute-1.amazonaws.com', fromhost-ip: 
'VNDR.INET-ADDR.HERE', HOSTNAME: '2017-08-19', PRI: 13,

syslogtag '21:', programname: '21', APP-NAME: '21', PROCID: '', MSGID: '-',
TIMESTAMP: 'Aug 19 21:28:52', STRUCTURED-DATA: '-',
msg: '20:13 [4166]   reason=feed.storage.hit type=event 
process_guid=0003--0ec0-01d3-1930ea50f35c segment_id=1 host='pc4' 
comms_ip='A.B.C.230' interface_ip='192.168.63.1' sensor_id=3 feed_id=14 
feed_name='virustotal' ioc_type='md5' 
ioc_value='c97f2ca4e0c49c2b79c955534b098c8f' timestamp='1503177613.09' 
start_time='2017-08-19T21:20:00.417Z' DATA-EDITED-BECAUSE-ITS-JUST-ALOT 
alliance_updated_virustotal='2017-07-26T22:26:48.000Z''



On 8/18/2017 4:38 PM, David Lang wrote:
templates define how rsyslog is going to output a line, but you are having 
problems because the data you are receiving is not formatted correctly and 
so the 'best effort' parsing is not giving you what you want.


log with the format RSYSLOG_DebugFormat and post an example of the message 
that is malformed. Look at the rawmsg section of that output. I'll bet that 
it will be fairly obvious what's wrong with the data you are being sent.


David Lang

 On Fri, 18 Aug 2017, Don M Subscriptions via rsyslog wrote:


Date: Fri, 18 Aug 2017 13:47:30 -0400
From: Don M Subscriptions via rsyslog <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Don M Subscriptions <donmrdch.subscripti...@gmail.com>
Subject: [rsyslog] Malfored syslog records missing host and extra date

Greetings – Posting again, don’t recall seeing if anyone could offer a
solution.

This time – index to the message.

1. Background – marked with *** BACKGROUND ***

2. Input sample – shows what we are receiving w/ a comparison line marked
with *** SAMPLE ***

3. rsyslog file (my receiver’s rsyslog file) – *** RSYSLOG ***

4. applicable template file (from third party vendor) *** TEMPLATE ***

*** BACKGROUND ***

We have a cloud based desktop analysis package that sends data when it
recognizes something
suspicious (suspicious connection, hit from a virus total feed, etc.). To
that end, the cloud system sends in data over TLS (which is working). I am
seeing a very odd pattern in the 'messages' file. I suspect that this
pattern or issue has to do w/ the template or definition assigned to data
received because there are multiple dates and no ‘sender’ 

Re: [rsyslog] Malfored syslog records missing host and extra date

[Don M Subscriptions via rsyslog]

First, thanks for the input!

There are two lines posted below. The first one is from the local 
system, its own IPtables record. To me, this sure looks right. The 
second one is the line from the external system, which is the syslog 
line that is incorrect. I did query the vendor, and it is thier 
direction that I "figure out how to get rid of or strip off my own 
rsyslog date". From what I can glean, this means that I need to create a 
"template" and then apply that specific template to data arriving from 
thier host. I did change a the data to sanitize it a bit. My changes are 
in UPPERCASE to make that clearer (I hope).


I think I need something like this:

# template to improve data from vendor cloud (Aug 21, 2017, DJM)
$template customFormat,"%HOSTNAME% %syslogtag%%msg%\n"
# $ActionFileDefaultTemplate customFormat
 RULES 
if $fromhost == 'ec2-VNDR.INET-ADDR.HERE.compute-1.amazonaws.com' then 
-/var/log/data-from-vendor;customFormat
if $fromhost-ip == 'VNDR.INET-ADDR.HERE' then 
-/var/log/data-from-vendor;customFormat



Debug line with all properties: NORMAL line.
FROMHOST: 'LOCALHOST_NAME_HERE', fromhost-ip: '127.0.0.1', HOSTNAME: 
'LOCALHOST_NAME_HERE', PRI: 6,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: 
'', MSGID: '-',

TIMESTAMP: 'Aug 19 21:28:51', STRUCTURED-DATA: '-',
msg: 'iptables: IN=eth0 OUT= 
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=VNDR.INET-ADDR.HERE 
DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=2154 DF 
PROTO=TCP SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0 '
escaped msg: 'iptables: IN=eth0 OUT= 
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=VNDR.INET-ADDR.HERE 
DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=2154 DF 
PROTO=TCP SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0 '
inputname: imklog rawmsg: 'iptables: IN=eth0 OUT= 
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=VNDR.INET-ADDR.HERE 
DST=LOCAL-IP.-HERE.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=2154 DF 
PROTO=TCP SPT=44208 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0 '


Debug line with all properties: SUSPECT-IMPROPER-UNHAPPY line
FROMHOST: 'ec2-VNDR.INET-ADDR.HERE.compute-1.amazonaws.com', 
fromhost-ip: 'VNDR.INET-ADDR.HERE', HOSTNAME: '2017-08-19', PRI: 13,

syslogtag '21:', programname: '21', APP-NAME: '21', PROCID: '', MSGID: '-',
TIMESTAMP: 'Aug 19 21:28:52', STRUCTURED-DATA: '-',
msg: '20:13 [4166]   reason=feed.storage.hit type=event 
process_guid=0003--0ec0-01d3-1930ea50f35c segment_id=1 
host='pc4' comms_ip='A.B.C.230' interface_ip='192.168.63.1' sensor_id=3 
feed_id=14 feed_name='virustotal' ioc_type='md5' 
ioc_value='c97f2ca4e0c49c2b79c955534b098c8f' timestamp='1503177613.09' 
start_time='2017-08-19T21:20:00.417Z' DATA-EDITED-BECAUSE-ITS-JUST-ALOT 
alliance_updated_virustotal='2017-07-26T22:26:48.000Z''



On 8/18/2017 4:38 PM, David Lang wrote:
templates define how rsyslog is going to output a line, but you are 
having problems because the data you are receiving is not formatted 
correctly and so the 'best effort' parsing is not giving you what you 
want.


log with the format RSYSLOG_DebugFormat and post an example of the 
message that is malformed. Look at the rawmsg section of that output. 
I'll bet that it will be fairly obvious what's wrong with the data you 
are being sent.


David Lang

 On Fri, 18 Aug 2017, Don M Subscriptions via rsyslog wrote:


Date: Fri, 18 Aug 2017 13:47:30 -0400
From: Don M Subscriptions via rsyslog 
To: rsyslog-users 
Cc: Don M Subscriptions 
Subject: [rsyslog] Malfored syslog records missing host and extra date

Greetings – Posting again, don’t recall seeing if anyone could offer a
solution.

This time – index to the message.

1. Background – marked with *** BACKGROUND ***

2. Input sample – shows what we are receiving w/ a comparison line 
marked

with *** SAMPLE ***

3. rsyslog file (my receiver’s rsyslog file) – *** RSYSLOG ***

4. applicable template file (from third party vendor) *** TEMPLATE ***

*** BACKGROUND ***

We have a cloud based desktop analysis package that sends data when it
recognizes something
suspicious (suspicious connection, hit from a virus total feed, 
etc.). To
that end, the cloud system sends in data over TLS (which is working). 
I am

seeing a very odd pattern in the 'messages' file. I suspect that this
pattern or issue has to do w/ the template or definition assigned to 
data

received because there are multiple dates and no ‘sender’ or ‘hostname’,
like in the *** SAMPLE *** section.

IPtables firewall line, which looks fine. Meaning all of the normal 
fields

are in place (key identifiers changed though with reasonable alternates.

Below that iptables line is a line received from the external host. I 
see
multiple dates, and I do not see the hostname in the second line, 
like I do
for My_Relay_Hostname (the obfuscated name of the rsyslog server 
itself).
What can cause 

Re: [rsyslog] Malfored syslog records missing host and extra date

[David Lang]

templates define how rsyslog is going to output a line, but you are having 
problems because the data you are receiving is not formatted correctly and so 
the 'best effort' parsing is not giving you what you want.


log with the format RSYSLOG_DebugFormat and post an example of the message that 
is malformed. Look at the rawmsg section of that output. I'll bet that it will 
be fairly obvious what's wrong with the data you are being sent.


David Lang

 On Fri, 18 Aug 
2017, Don M Subscriptions via rsyslog wrote:



Date: Fri, 18 Aug 2017 13:47:30 -0400
From: Don M Subscriptions via rsyslog 
To: rsyslog-users 
Cc: Don M Subscriptions 
Subject: [rsyslog] Malfored syslog records missing host and extra date

Greetings – Posting again, don’t recall seeing if anyone could offer a
solution.

This time – index to the message.

1. Background – marked with *** BACKGROUND ***

2. Input sample – shows what we are receiving w/ a comparison line marked
with *** SAMPLE ***

3. rsyslog file (my receiver’s rsyslog file) – *** RSYSLOG ***

4. applicable template file (from third party vendor) *** TEMPLATE ***

*** BACKGROUND ***

We have a cloud based desktop analysis package that sends data when it
recognizes something
suspicious (suspicious connection, hit from a virus total feed, etc.). To
that end, the cloud system sends in data over TLS (which is working). I am
seeing a very odd pattern in the 'messages' file. I suspect that this
pattern or issue has to do w/ the template or definition assigned to data
received because there are multiple dates and no ‘sender’ or ‘hostname’,
like in the *** SAMPLE *** section.

IPtables firewall line, which looks fine. Meaning all of the normal fields
are in place (key identifiers changed though with reasonable alternates.

Below that iptables line is a line received from the external host. I see
multiple dates, and I do not see the hostname in the second line, like I do
for My_Relay_Hostname (the obfuscated name of the rsyslog server itself).
What can cause multiple dates? Is there some way that I can enforce storing
the sending IP and/or the sending system host name, which is missing
(second line, again)?

*** SAMPLE ***

IPtables local firewall output – date, localhostname, and then the message.
IPtables firewall line, which looks fine. Meaning all of the normal fields
are in place (key identifiers changed though with reasonable alternates.

Aug  7 13:08:54 My_Relay_Hostname kernel: iptables: IN=eth0 OUT=
MAC=00:0c:29:3c:b6:61:58:49:3b:14:5d:14:08:00 SRC=AA.BB.CC.30
DST=DD.EE.FF.176 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=18235 DF PROTO=TCP
SPT=58752 DPT=10514 WINDOW=17922 RES=0x00 SYN URGP=0

Cloud monitoring solution – date, date repeated, no local hostname (missing)

Aug  7 13:08:55 2017-08-07 16: 50:16 [4166]   reason=watchlist.hit
type=event process_guid=0007--0f8c-01d3-0f95444b7a13 segment_id=1
host='victim_hostname_here' comms_ip='GGG.HHH.III.202'
interface_ip='DD.JJ.KK.112' sensor_id=7 watchlist_id=280
watchlist_name='Notepad' timestamp='1502124615.77'
start_time='2017-08-07T15:53:09.421Z' group='testing_data_group_name_here'
process_md5='sum_value_here' process_name='not.exe'
process_path='c:\windows\system32\not.exe'
last_update='2017-08-07T16:46:35.818Z'

*** TEMPLATE *** From what I can tell from the rsyslog file, this is the
definition for received data external to the rsyslog receiving host

# this module manages the delivery of syslog calls from a logging process
to rsyslog
#$ModLoad imuxsock.so
# imklog: Reads messages from the kernel log and submits them to the syslog
engine.
#$ModLoad imklog.so
#$ActionFileDefaultTemplate  RSYSLOG_TraditionalFileFormat
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none/var/log/messages

# Remote logging
#$ModLoad imudp
#$UDPServerAddress 0.0.0.0
#$UDPServerRun 514
#Dyanmic folders
$template DYNmessages,"/var/log/VENDOR/devices/%fromhost-ip%/messages"
$template DYNsecure,"/var/log/VENDOR/devices/%fromhost-ip%/secure"
$template DYNmaillog,"/var/log/VENDOR/devices/%fromhost-ip%/maillog"
$template DYNcron,"/var/log/VENDOR/devices/%fromhost-ip%/cron"
$template DYNspooler,"/var/log/VENDOR/devices/%fromhost-ip%/spooler"
$template DYNboot,"/var/log/VENDOR/devices/%fromhost-ip%/boot.log"
$template DYNlog,"/var/log/VENDOR/devices/%fromhost-ip%/%fromhost-ip%.log"
if \
   $fromhost-ip != '127.0.0.1' \
   and \
   $fromhost-ip != '172.17.0.29' \
then?DYNlog

*** RSYSLOG *** Then there is the default rsyslog.conf file, a wee bit
sanitized.

$MaxMessageSize 64k
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support