Kibana just renders the data, so I have no specific configuration for

My logstash config (mostly cribbed from is as follows:



input {
  syslog {
    type => syslog
    port => 5544
  udp {
    type => syslogjson
    port => 5500
    codec => "json"

filter {
  # This replaces the host field (UDP source) with the host that
generated the message (sysloghost)
  if [sysloghost] {
    mutate {
      replace => [ "host", "%{sysloghost}" ]
      remove_field => "sysloghost" # prune the field after successfully
replacing "host"

output {
  elasticsearch {
    protocol => node
    node_name => "Indexer01"

This is my dev cluster which runs a logstash-1.4.1 RPM install
connecting to an elasticsearch cluster running on 3 workstations and a
laptop.  The UDP connection is only used by a single client, so could be
ignored.  This is the JSON sending that I referred to previously.  Not
entirely successful so far.

On my "prod" system I've also managed to write some grok filters:


input {
  syslog {
    type => syslog
    port => 5544

filter {
  if [type] == "syslog" {
    grok {
      patterns_dir => "/opt/logstash/patterns"
      match => { "message" => "%{BESPOKFW}" }
      match => { "message" => "%{AUDITAVC}" }

output {
  elasticsearch {
    embedded => true
    template_overwrite => true
    manage_template => false

/opt/logstash/patterns/bespokfw containing

ETHTYPE (?:(?:[A-Fa-f0-9]{2}):(?:[A-Fa-f0-9]{2}))
IPTABLES1 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}
IPTABLES2 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}


/opt/logstash/patterns/auditavc containing

AVCDEV (%{NUMBER:devmaj}:%{NUMBER:devmin})
AUDITAVC (?:type=%{WORD:audit_type}
avc:\s*%{WORD:avc_action}\s*\{ %{WORD:avc_type} \}
for\s*pid=(%{NUMBER:avc_pid})? comm=\"(%{WORD:avc_comm})?\"
dev=(%{AVCDEV:avc_dev})? ino=(%{NUMBER:avc_ino})?)?
tcontext=(%{NOTSPACE:avc_tcontext})? tclass=(%{WORD:avc_tclass})?)

This is running a tarball version of logstash (1.3.3 I think) with an
embedded elasticsearch instance.

Both work reasonably well.  Am looking to bring more log data back at
the moment (i.e. application specific logs).



> -----Original Message-----
> From: Josh [] 
> Sent: 03 June 2014 11:54
> To: Innes, Duncan
> Cc: freeipa-users
> Subject: Re: [Freeipa-users] Setting up IPA to log remotely
> On Jun 3, 2014, at 4:37 AM, Innes, Duncan 
> <> wrote:
> > I'm starting to log IPA to a central point too.  I'd hoped 
> the A part 
> > of IPA would have arrived, but other functionality has 
> pushed it down 
> > the priority list.  Would be good to see it arrive as something 
> > integrated with systemd/journald with fully separated log fields 
> > instead of a simple log text line.
> > 
> > For now, rsyslog does a decent job of sending the logs over the 
> > network and I'm using logstash to parse logs and pop them into 
> > elasticsearch for analysing via Kibana.  I've had most trouble with 
> > the rsyslog side of things, but that's because I tried to 
> get rsyslog 
> > to send in JSON format rather than plain text.  Once I 
> reigned in my 
> > ambition, it proved to be somewhat easier -
> > 
> Any chance you could share your kibana configuration?
> > All I've added to RHEL6 client is a file 
> /etc/rsyslog.d/logstash.conf 
> > with contents:
> > 
> > *.*
> > 
> > and (firewalls permitting) my logs end up at the logstash 
> server for 
> > parsing.
> > 
> > Duncan
> <snip>
> -josh
> This message has been checked for viruses and spam by the 
> Virgin Money email scanning system powered by Messagelabs.

This message has been checked for viruses and spam by the Virgin Money email 
scanning system powered by Messagelabs.

This e-mail is intended to be confidential to the recipient. If you receive a 
copy in error, please inform the sender and then delete this message.

Virgin Money plc - Registered in England and Wales (Company no. 6952311). 
Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL. 
Virgin Money plc is authorised by the Prudential Regulation Authority and 
regulated by the Financial Conduct Authority and the Prudential Regulation 

The following companies also trade as Virgin Money. They are both authorised 
and regulated by the Financial Conduct Authority, are registered in England and 
Wales and have their registered office at Jubilee House, Gosforth, Newcastle 
upon Tyne NE3 4PL: Virgin Money Personal Financial Service Limited (Company no. 
3072766) and Virgin Money Unit Trust Managers Limited (Company no. 3000482).

For further details of Virgin Money group companies please visit our website at

Freeipa-users mailing list

Reply via email to