Kibana just renders the data, so I have no specific configuration for
that.
My logstash config (mostly cribbed from logstash.net) is as follows:
/etc/logstash/conf.d/syslog.conf
Containing:
input {
syslog {
type => syslog
port => 5544
}
udp {
type => syslogjson
port => 5500
codec => "json"
}
}
filter {
# This replaces the host field (UDP source) with the host that
generated the message (sysloghost)
if [sysloghost] {
mutate {
replace => [ "host", "%{sysloghost}" ]
remove_field => "sysloghost" # prune the field after successfully
replacing "host"
}
}
}
output {
elasticsearch {
protocol => node
node_name => "Indexer01"
}
}
This is my dev cluster which runs a logstash-1.4.1 RPM install
connecting to an elasticsearch cluster running on 3 workstations and a
laptop. The UDP connection is only used by a single client, so could be
ignored. This is the JSON sending that I referred to previously. Not
entirely successful so far.
On my "prod" system I've also managed to write some grok filters:
/etc/logstash.conf
input {
syslog {
type => syslog
port => 5544
}
}
filter {
if [type] == "syslog" {
grok {
patterns_dir => "/opt/logstash/patterns"
match => { "message" => "%{BESPOKFW}" }
match => { "message" => "%{AUDITAVC}" }
}
}
}
output {
elasticsearch {
embedded => true
template_overwrite => true
manage_template => false
}
}
With
/opt/logstash/patterns/bespokfw containing
NETFILTERMAC
%{COMMONMAC:dst_mac}:%{COMMONMAC:src_mac}:%{ETHTYPE:ethtype}
ETHTYPE (?:(?:[A-Fa-f0-9]{2}):(?:[A-Fa-f0-9]{2}))
IPTABLES1 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}
DST=%{IP:dst_ip}.*PROTO=%{WORD:proto}?.*SPT=%{INT:src_port}?.*DPT=%{INT:
dst_port}?.*)
IPTABLES2 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}
DST=%{IP:dst_ip}.*PROTO=%{INT:proto}?.*)
BESPOKFW (?:%{IPTABLES1}|%{IPTABLES2})
And
/opt/logstash/patterns/auditavc containing
AVCDEV (%{NUMBER:devmaj}:%{NUMBER:devmin})
AUDITAVC (?:type=%{WORD:audit_type}
audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\):
avc:\s*%{WORD:avc_action}\s*\{ %{WORD:avc_type} \}
for\s*pid=(%{NUMBER:avc_pid})? comm=\"(%{WORD:avc_comm})?\"
%{WORD:avc_class}=\"(%{NOTSPACE:avc_class_value})?\"(
dev=(%{AVCDEV:avc_dev})? ino=(%{NUMBER:avc_ino})?)?
scontext=(%{NOTSPACE:avc_scontext})?
tcontext=(%{NOTSPACE:avc_tcontext})? tclass=(%{WORD:avc_tclass})?)
This is running a tarball version of logstash (1.3.3 I think) with an
embedded elasticsearch instance.
Both work reasonably well. Am looking to bring more log data back at
the moment (i.e. application specific logs).
Cheers
Duncan
> -----Original Message-----
> From: Josh [mailto:joka...@gmail.com]
> Sent: 03 June 2014 11:54
> To: Innes, Duncan
> Cc: freeipa-users
> Subject: Re: [Freeipa-users] Setting up IPA to log remotely
>
>
> On Jun 3, 2014, at 4:37 AM, Innes, Duncan
> <duncan.in...@virginmoney.com> wrote:
>
> > I'm starting to log IPA to a central point too. I'd hoped
> the A part
> > of IPA would have arrived, but other functionality has
> pushed it down
> > the priority list. Would be good to see it arrive as something
> > integrated with systemd/journald with fully separated log fields
> > instead of a simple log text line.
> >
> > For now, rsyslog does a decent job of sending the logs over the
> > network and I'm using logstash to parse logs and pop them into
> > elasticsearch for analysing via Kibana. I've had most trouble with
> > the rsyslog side of things, but that's because I tried to
> get rsyslog
> > to send in JSON format rather than plain text. Once I
> reigned in my
> > ambition, it proved to be somewhat easier -
> >
>
> Any chance you could share your kibana configuration?
> > All I've added to RHEL6 client is a file
> /etc/rsyslog.d/logstash.conf
> > with contents:
> >
> > *.* @logstash.example.com:5544
> >
> > and (firewalls permitting) my logs end up at the logstash
> server for
> > parsing.
> >
> > Duncan
> <snip>
>
> -josh
>
> This message has been checked for viruses and spam by the
> Virgin Money email scanning system powered by Messagelabs.
>
This message has been checked for viruses and spam by the Virgin Money email
scanning system powered by Messagelabs.
This e-mail is intended to be confidential to the recipient. If you receive a
copy in error, please inform the sender and then delete this message.
Virgin Money plc - Registered in England and Wales (Company no. 6952311).
Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL.
Virgin Money plc is authorised by the Prudential Regulation Authority and
regulated by the Financial Conduct Authority and the Prudential Regulation
Authority.
The following companies also trade as Virgin Money. They are both authorised
and regulated by the Financial Conduct Authority, are registered in England and
Wales and have their registered office at Jubilee House, Gosforth, Newcastle
upon Tyne NE3 4PL: Virgin Money Personal Financial Service Limited (Company no.
3072766) and Virgin Money Unit Trust Managers Limited (Company no. 3000482).
For further details of Virgin Money group companies please visit our website at
virginmoney.com
_______________________________________________
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users
