what version of rsyslog are you using (so I have a clue how old the 'official
document' you are following is). This would be far from the first time that ES
has changed the requirements on us.
But this clearly shows the problem ES ddoesn't support text/json; charset=utf8.
But I don't know what they do support.
what version of ES are you connecting to?
David Lang
On Fri, 27 Mar 2020, 来自小七and雨 via
rsyslog wrote:
Date: Fri, 27 Mar 2020 09:38:36 +0800 (CST)
From: 来自小七and雨 via rsyslog <[email protected]>
To: "[email protected]" <[email protected]>
Cc: 来自小七and雨 <[email protected]>
Subject: Re: [rsyslog] rsyslog output to elasticsearch doesn't works
Thanks David Lang.
Now I get an error msg:
{ "request": { "url": "http:\/\/manager.server:9200\/test-index\/test-type",
"postdata": "{\"message\":\"Unregistered Authentication Agent for unix-process:12318:17143977 (system bus name
:1.345163, object path \\\/org\\\/freedesktop\\\/PolicyKit1\\\/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)\",\"fromhost\":\"master\",\"facility\":\"authpriv\",\"priority\
":\"notice\",\"timereported\":\"2020-03-27T09:33:46.020173+08:00\",\"timegenerated\":\"2020-03-27T09:33:46.020173+08:00\"}" },
"reply": { "error": "Content-Type header [text\/json; charset=utf-8] is not supported", "status": 406 } }
_________________________________________________________________________
"Content-Type header [text\/json; charset=utf-8] is not supported", "status": 406
I used the template of the official document. Is there a problem?
At 2020-03-27 09:22:35, "来自小七and雨 via rsyslog" <[email protected]>
wrote:
All Config：
——————————————————————————————————
# rsyslog configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of
imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger
command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
#module(load="imfile") #needs to be done just once
module(load="imfile" PollingInterval="1")
module(load="omkafka")
module(load="omelasticsearch")
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$template myFormat,"%timestamp% %fromhost-ip% %msg%\n"
$ActionFileDefaultTemplate myFormat
template(name="testTemplate"
type="list"
option.json="on") {
constant(value="{")
constant(value="\"timestamp\":\"") property(name="timereported"
dateFormat="rfc3339")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"")
property(name="syslogseverity-text")
constant(value="\",\"facility\":\"")
property(name="syslogfacility-text")
constant(value="\",\"syslogtag\":\"") property(name="syslogtag")
constant(value="\"}")
}
# File syncing capability is disabled by default. This feature is usually not
required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on
# File to store the position in the journal
$IMJournalStateFile imjournal.state
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.* /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
input(type="imfile" File="/var/log/app.his.log" Tag="user-cmd" Severity="info"
Facility="local1")
*.info;mail.none;authpriv.none;cron.none @@info.server.com:514
local1.info/data/log/testkafka
&action(type="omkafka" topic="mytopic" confParam="compression.codec=snappy"
broker="manager.server:9092")
local1.info action(type="omelasticsearch" server="manager.server:9200"
searchIndex="test-index" searchType="test-type")
_____________________________________________________________________________________________
At 2020-03-27 08:37:07, "来自小七and雨 via rsyslog" <[email protected]>
wrote:
sorry,here is config:
__________________________________________________
module(load="omkafka")
module(load="omelasticsearch")
template(name="testTemplate"
type="list"
option.json="on") {
constant(value="{")
constant(value="\"timestamp\":\"") property(name="timereported"
dateFormat="rfc3339")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"")
property(name="syslogseverity-text")
constant(value="\",\"facility\":\"")
property(name="syslogfacility-text")
constant(value="\",\"syslogtag\":\"") property(name="syslogtag")
constant(value="\"}")
}
local1.info action(type="omelasticsearch" server="manager.server:9200"
searchIndex="test-index" searchType="test-type")
___________________________________________________________
And, there is no error log。
I tried "rsyslogd -n" startup, but there was no extra information output, no
error was reported, and elasticsearch did not receive the messages. This confuses me.
Also, I used it to forward the message to kafka's message successfully.
Any suggestions
thanks
At 2020-03-27 01:04:38, "John Chivian via rsyslog" <[email protected]>
wrote:
No one can help you unless you provide detail. Start with your exact
rsyslog configuration, and any examples of error messages.
Regards,
On 3/26/20 5:34 AM, 来自小七and雨 via rsyslog wrote:
Hi everyone,
I tried using rsyslog to send log messages to es, but failed.
I checked that the IP and port of es are correct, and I have also confirmed
that the es plugins is installed.
Checking that No corresponding index/type was created in es..
Can anyone help me? Thank you!
______________________________________
env :
elasticsearch v7.3
rsyslog v8.24
centos v7.4
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
