Attached is a full debug log (hostnames and dir names scrubbed). But the basic
setup is this - rsyslog listens on UDP, plain TCP, TLS and RELP ports, parses
the syslogtag and outputs to a Kafka topic of the same name via omkafka. UDP,
TCP and RELP work as intended, and the issue occurs on both 8.22 and 8.23. In
my opinion it’s likely a configuration issue
My configuration is setup in a straight forward include structure, my main
rsyslog.conf file looks like:
$IncludeConfig
/<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/globals/*.conf
$IncludeConfig
/<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/modules/*.conf
$IncludeConfig
/<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/templates/*.conf
$IncludeConfig /<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/inputs/*.conf
$IncludeConfig /<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/rules/*.conf
$IncludeConfig /<app_dir>/managed_by_ansible/rsyslog/etc/rsyslog.d/*.conf
I have a number of small conf files from there:
./rsyslog.conf
./rsyslog.d
./rsyslog.d/globals
./rsyslog.d/globals/01_common.conf
./rsyslog.d/globals/02_main_queue.conf
./rsyslog.d/inputs
./rsyslog.d/inputs/01_udp.conf
./rsyslog.d/inputs/02_tcp.conf
./rsyslog.d/inputs/03_tls.conf
./rsyslog.d/inputs/04_relp.conf
./rsyslog.d/modules
./rsyslog.d/modules/01_imudp.conf
./rsyslog.d/modules/02_imtcp.conf
./rsyslog.d/modules/03_imtls.conf
./rsyslog.d/modules/04_imrelp.conf
./rsyslog.d/modules/05_imfile.conf
./rsyslog.d/modules/09_impstats.conf
./rsyslog.d/modules/20_omkafka.conf
./rsyslog.d/rules
./rsyslog.d/rules/00_rsyslogd_internal.conf
./rsyslog.d/rules/01_udp2kafka.conf
./rsyslog.d/rules/02_tcp2kafka.conf
./rsyslog.d/rules/03_tls2kafka.conf
./rsyslog.d/rules/04_relp2kafka.conf
./rsyslog.d/rules/50_sawmill_stats.conf
./rsyslog.d/templates
./rsyslog.d/templates/kakfa_json.conf
./rsyslog.d/templates/local_files.conf
The dump occurs during the loading of the TLS portion (imtcp), here’s the conf
files for the TLS config (scrubbed):
03_tls.conf:
input(
type = "imtcp"
port = "10448"
ruleset = "tls_10448"
)
03_imtls.conf:
module(
load = "imtcp"
maxSessions = "300"
StreamDriver.Name = "gtls"
StreamDriver.Mode = "1"
StreamDriver.AuthMode = "x509/name"
PermittedPeer = [ "*.apple.com", “*.----.apple.com",
“*.----.apple.com" ]
)
03_tls2kafka.conf:
ruleset(name = "tls_10448") {
if $syslogtag startswith "sawmill_test_topic_tls" then {
action(
name = "tls10448_sawmill_test_topic_tls_to_kafka"
type = "omkafka"
# Queue configuration - in-memory up to 976.5625M, then to disk up
to 1024M
queue.spoolDirectory =
“/<app_dir>/managed_by_ansible/rsyslog/var/spool"
queue.type = "LinkedList"
queue.size = "1000000"
queue.filename = "tls10448_sawmill_test_topic_tls_to_kafka"
queue.maxdiskspace = "1024M"
queue.saveonshutdown = "off"
queue.highwatermark = "900000"
queue.lowwatermark = "400000"
queue.discardseverity = "5"
# Kafka configuration
template = "KAFKA_JsonLineTemplate"
broker = [ “broker1:9092", “broker2:9092",
“broker3:9092", “broker4:9092", “broker5.corp.apple.com:9092" ]
confParam = ["queue.buffering.max.messages=10000"]
topic = "sawmill_test_topic_tls"
partitions.auto = "on"
)
stop
}
}
Andrew Griffin
ETS / Integration Services
☏ 408-783-8348
> On Dec 21, 2016, at 12:05 AM, mostolog--- via rsyslog
> <[email protected]> wrote:
>
> It will be great if you provide a replication use case:
>
> * rsyslog version (may be 8.23)
> * configuration file
> * log example to replicate the issue.
>
> If you are able to replicate the issue with a "not too old" syslog version,
> please file an issue on github
>
>
> El 20/12/16 a las 21:14, Andrew Griffin via rsyslog escribió:
>> I’m working setting up a syslog pipeline with rsyslog at the front, and I’m
>> running in to an issue with rsyslogd core dumping when I try to start it.
>> Debug log shows everything loading fine, then this:
>>
>> 4473.240773513:imtcp.c : nspoll.c:147 ISOBJ assert failure: invalid
>> object type, expected 'netstrms' actual 'nspoll', cookie: BADEFEE
>> rsyslogd: nspoll.c:147: SetDrvrName: Assertion `0' failed.
>>
>> Then it core dumps. Is this a known issue? I can provide a full debug log
>> if necessary
>>
>> Andrew Griffin
>> ETS / Integration Services
>> ☏ 408-783-8348
>>
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
