On 06/02/2015 10:29 AM, Innes, Duncan wrote:
Just a bit of a head's up and a refresh of this with perhaps some new
data.
Good to hear :-) We recently also started investigating the Audit
capabilities for (notice I write "for" and not "in") IPA. You can
check my initial nudge to the freeipa-users list, which was
unfortunately with no reply:
https://www.redhat.com/archives/freeipa-users/2015-March/msg00940.html
First up, just got round to reading this Martin. Not sure how I missed
it
when it first came out as it's a strong area of interest for me.
We have more now! I created
https://www.freeipa.org/page/Centralized_Logging
where you can read more about this POC project and even see demo showing what
the current POC ELK server can do (with link to Docker server container and
sources of course).
The main part of what this message is about is a big change I made to
our
logging recently. I added in 4 of our main production IPA servers
(there
are 8 in total, but 4 sit beyond firewalls that take more scrutiny for
changes than I wanted for now). The 4 I've added, though, serve more
clients I figure. The amount of log traffic to the pair of Logstash
servers has now jumped from around 50k records/hour to around 250k.
Doubtless this still doesn't push any of the parts to their limits, but
there has been a barely noticeably increase in CPU usage on the 2
Logstash
servers. We've gone from around 2% CPU usage to 4%.
Since the CPU usage on our 'loudest' IPA server rarely peaks above 10%,
this doesn't present nearly as much load as I had anticipated. I have
run
Logstash parsers on my DEV IPA boxes, but will now investigate running
them on my Prod servers too.
Cool!
What I'm getting at is that perhaps clients sending logs back to the IPA
servers for parsing, then being sent on to a central DB for storage,
isn't
going to break the bank performance-wise.
All of the systems in question here are 2vCPU with 4Gb vRAM running on
ESXi
hosts, so nothing special in the performance arena.
It strikes me as a reasonably elegant solution to pair the
authentication
and log parsing services on the same set of servers. This would allow
each
client to use the same servers/failover etc for SSSD as for rsyslog.
There may, of course, be other considerations, but I'm suggesting that
system load isn't necessarily one of them. Much as projects such as
Katello
can run with everything on the same server, or split out Postgres and
the
like onto separate servers when there are performance considerations.
Thoughts? I'm not saying they should always be paired, but that if a
user
designs a system with enough horse power, this piggy-backing could work
well.
Ah, interesting idea and measurement. We have not thought about this kind of
architecture yet. What we did in our POC is to configure FreeIPA clients and
servers to send the logs directly to the logging server which was on completely
different machine (container) than rest of the infrastructure.
It may be an alternative scheme, to have FreeIPA server containing the log
processing and then forwarding further to other REK/ELK server and clients
simply forwarding the logs to the same server as where they are authenticating.
If all the log configuration is baked in ipa-{server,replica,client}-install,
it would be extremely easy to integrate.
I am not sure if the authentication+logging binding would be that easy, nor
that it belong together that much. SSSD would need to dynamically export the
address of the FreeIPA server it communicates with, maybe similarly as it does
with Kerberos (http://linux.die.net/man/8/sssd_krb5_locator_plugin) - but that
does not seem as a good fit either.
In any case, CCing Jakub for reference.
Thanks,
Martin
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
