Re: [Freeipa-users] Setting up IPA to log remotely
I'm starting to log IPA to a central point too. I'd hoped the A part of IPA would have arrived, but other functionality has pushed it down the priority list. Would be good to see it arrive as something integrated with systemd/journald with fully separated log fields instead of a simple log text line. For now, rsyslog does a decent job of sending the logs over the network and I'm using logstash to parse logs and pop them into elasticsearch for analysing via Kibana. I've had most trouble with the rsyslog side of things, but that's because I tried to get rsyslog to send in JSON format rather than plain text. Once I reigned in my ambition, it proved to be somewhat easier - All I've added to RHEL6 client is a file /etc/rsyslog.d/logstash.conf with contents: *.* @logstash.example.com:5544 and (firewalls permitting) my logs end up at the logstash server for parsing. Duncan -Original Message- From: freeipa-users-boun...@redhat.com [mailto:freeipa-users-boun...@redhat.com] On Behalf Of Brendan Kearney Sent: 03 June 2014 03:26 To: Steven Jones Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] Setting up IPA to log remotely On Tue, 2014-06-03 at 00:42 +, Steven Jones wrote: Hi, I'll raise a request for this to be added then. Its a bit of an enterprise requirement feature that is of use for us. Not having much luck with rsyslog and application logs at the moment, good and accurate docs seem lacking for RHEL6. regards Steven From: Rob Crittenden rcrit...@redhat.com Sent: Tuesday, 3 June 2014 9:27 a.m. To: Steven Jones Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] Setting up IPA to log remotely Steven Jones wrote: Is there a way to get IPA to send its logs remotely? We intend to do something like this with audit, most likely using the systemd journal, but it's a ways off. For now you'd need to do it manually on a per-service basis. I'd suggest looking at rsyslogd. You should be able to at least get the Apache and 389-ds logs using that. rob ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users check out http://www.rsyslog.com/doc/master/index.html for good and accurate docs. i am using fedora 16 and 20 with RELP, fowarding syslog from everywhere to a central location, and then dumping the logs into mysql. phplogcon bolts on top of it for a web view of all the logs. on a sending source: $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $SystemLogRateLimitInterval 0 $IMUXSockRateLimitInterval 0 $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # Provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 514 # Provides RELP transmission $ModLoad omrelp *.* :omrelp:192.168.25.1:20514;RSYSLOG_ForwardFormat ~ on a receiving destination: $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $SystemLogRateLimitInterval 0 $IMUXSockRateLimitInterval 0 $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # Provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 514 # Provides RELP reception $ModLoad imrelp $InputRELPServerRun 20514 # Provides MySQL connectivity $ModLoad ommysql # MASSIVE INSERT RATE FOR DB / SCALED DB LOGGING $WorkDirectory /var/spool/rsyslog # default location for work (spool) files $ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName dbq# set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure # for PostgreSQL replace :ommysql: by :ompgsql: below: *.* :ommysql:server.domain.tld,Syslog,user,password ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users This message has been checked for viruses and spam by the Virgin Money email scanning system powered by Messagelabs. This message has been checked for viruses and spam by the Virgin Money email scanning system powered by Messagelabs. This e-mail is intended to be confidential to the recipient. If you receive a copy in error, please inform the sender and then delete this message. Virgin Money plc - Registered in England and Wales (Company no. 6952311). Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL. Virgin Money plc is authorised by the Prudential Regulation Authority and regulated by the
Re: [Freeipa-users] Setting up IPA to log remotely
Kibana just renders the data, so I have no specific configuration for
that.
My logstash config (mostly cribbed from logstash.net) is as follows:
/etc/logstash/conf.d/syslog.conf
Containing:
input {
syslog {
type = syslog
port = 5544
}
udp {
type = syslogjson
port = 5500
codec = json
}
}
filter {
# This replaces the host field (UDP source) with the host that
generated the message (sysloghost)
if [sysloghost] {
mutate {
replace = [ host, %{sysloghost} ]
remove_field = sysloghost # prune the field after successfully
replacing host
}
}
}
output {
elasticsearch {
protocol = node
node_name = Indexer01
}
}
This is my dev cluster which runs a logstash-1.4.1 RPM install
connecting to an elasticsearch cluster running on 3 workstations and a
laptop. The UDP connection is only used by a single client, so could be
ignored. This is the JSON sending that I referred to previously. Not
entirely successful so far.
On my prod system I've also managed to write some grok filters:
/etc/logstash.conf
input {
syslog {
type = syslog
port = 5544
}
}
filter {
if [type] == syslog {
grok {
patterns_dir = /opt/logstash/patterns
match = { message = %{BESPOKFW} }
match = { message = %{AUDITAVC} }
}
}
}
output {
elasticsearch {
embedded = true
template_overwrite = true
manage_template = false
}
}
With
/opt/logstash/patterns/bespokfw containing
NETFILTERMAC
%{COMMONMAC:dst_mac}:%{COMMONMAC:src_mac}:%{ETHTYPE:ethtype}
ETHTYPE (?:(?:[A-Fa-f0-9]{2}):(?:[A-Fa-f0-9]{2}))
IPTABLES1 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}
DST=%{IP:dst_ip}.*PROTO=%{WORD:proto}?.*SPT=%{INT:src_port}?.*DPT=%{INT:
dst_port}?.*)
IPTABLES2 (?:%{WORD:vmfw}: IN=(%{WORD:in_device})?
OUT=(%{WORD:out_device})? (MAC=%{NETFILTERMAC})?.*SRC=%{IP:src_ip}
DST=%{IP:dst_ip}.*PROTO=%{INT:proto}?.*)
BESPOKFW (?:%{IPTABLES1}|%{IPTABLES2})
And
/opt/logstash/patterns/auditavc containing
AVCDEV (%{NUMBER:devmaj}:%{NUMBER:devmin})
AUDITAVC (?:type=%{WORD:audit_type}
audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\):
avc:\s*%{WORD:avc_action}\s*\{ %{WORD:avc_type} \}
for\s*pid=(%{NUMBER:avc_pid})? comm=\(%{WORD:avc_comm})?\
%{WORD:avc_class}=\(%{NOTSPACE:avc_class_value})?\(
dev=(%{AVCDEV:avc_dev})? ino=(%{NUMBER:avc_ino})?)?
scontext=(%{NOTSPACE:avc_scontext})?
tcontext=(%{NOTSPACE:avc_tcontext})? tclass=(%{WORD:avc_tclass})?)
This is running a tarball version of logstash (1.3.3 I think) with an
embedded elasticsearch instance.
Both work reasonably well. Am looking to bring more log data back at
the moment (i.e. application specific logs).
Cheers
Duncan
-Original Message-
From: Josh [mailto:joka...@gmail.com]
Sent: 03 June 2014 11:54
To: Innes, Duncan
Cc: freeipa-users
Subject: Re: [Freeipa-users] Setting up IPA to log remotely
On Jun 3, 2014, at 4:37 AM, Innes, Duncan
duncan.in...@virginmoney.com wrote:
I'm starting to log IPA to a central point too. I'd hoped
the A part
of IPA would have arrived, but other functionality has
pushed it down
the priority list. Would be good to see it arrive as something
integrated with systemd/journald with fully separated log fields
instead of a simple log text line.
For now, rsyslog does a decent job of sending the logs over the
network and I'm using logstash to parse logs and pop them into
elasticsearch for analysing via Kibana. I've had most trouble with
the rsyslog side of things, but that's because I tried to
get rsyslog
to send in JSON format rather than plain text. Once I
reigned in my
ambition, it proved to be somewhat easier -
Any chance you could share your kibana configuration?
All I've added to RHEL6 client is a file
/etc/rsyslog.d/logstash.conf
with contents:
*.* @logstash.example.com:5544
and (firewalls permitting) my logs end up at the logstash
server for
parsing.
Duncan
snip
-josh
This message has been checked for viruses and spam by the
Virgin Money email scanning system powered by Messagelabs.
This message has been checked for viruses and spam by the Virgin Money email
scanning system powered by Messagelabs.
This e-mail is intended to be confidential to the recipient. If you receive a
copy in error, please inform the sender and then delete this message.
Virgin Money plc - Registered in England and Wales (Company no. 6952311).
Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL.
Virgin Money plc is authorised by the Prudential Regulation Authority and
regulated by the Financial Conduct Authority and the Prudential Regulation
Authority.
The following companies also trade as Virgin Money. They are both authorised
and regulated by the Financial Conduct Authority, are registered in England and
Wales and have their registered office at Jubilee House, Gosforth, Newcastle
upon
[Freeipa-users] IPA+AD trust and NFS nobody issue
Hi, Environment: RHEL 7 IPA Server 3.3 with a trust to a Windows 2012 Server AD RHEL 7 NFS Server RHEL 7 Client I have found one problem when using a NFS 4 shared Home Directory for AD users logging in to IPA. I have created a NFS share /home/adexample.org and use autofs map in IPA. All wbinfo tests works as well as id. I can login fine through SSH and Shell with adt...@adexample.org The problem is that I can add the AD user as owner of his Home Directory and if I log in to the NFS Server locally or through ssh permissions are correct but when logging in to any other computer i get nobody as owner. Groups are no problem since AD groups can be mapped to Posix groups. Idmap.conf domain is set to the IPA Domain. Is there some way to get NFS working with the AD user as owner of his Home Directory? Thanks for any help. This e-mail is private and confidential between the sender and the addressee. In the event of misdirection, the recipient is prohibited from using, copying or disseminating it or any information in it. Please notify the above if any misdirection. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Getting Samba3 and FreeIPAv3 working together
Hi Petr Sumit, I've been trying to get further with my setup. 1. Thanks Petr, the groups.js plugin seems to work fine, it shows the correct info on the GUI screen and seems to be ok. 2. Sumit, I'm afraid that I'm having a few more problems after running ipa-adtrust-install --add-sids. I cannot now add any users on the server (Fedora 20, ipa-server 3.3.5-1) via the command-line or GUI. I get the following error: GUI: IPA Error 4205 missing attribute: sambaSID required by object class sambaSamAccount Command-line: ipa user-add test1234 . ipa: ERROR: missing attribute sambaSID required by object class sambaSamAccount Also, when editing an existing user, there is no sambaSID field available to edit. If you have any ideas, please let me know. Thanks, Dylan. On 26 May 2014 11:40, Petr Vobornik pvobo...@redhat.com wrote: On 23.5.2014 16:31, Dylan Evans wrote: Hi Sumit and Petr, Thanks both of you for your replies, I've now got to go and try to implement all your suggestions but I have some more questions, sorry! The guide at techslaves was fine, I just got stuck with the changes in the JavaScript packages and the Samba server questions. 1. Petr, I put your samba.js plugin into /usr/share/ipa/ui/js/plugins/samba but you'll have to pardon my lack of JS knowledge, anything more than simple Bash scripts tends to leave me confused! Do I need to do anything else apart from restart the IPA service? I read your info at http://pvoborni.fedorapeople.org/doc/#!/guide/Plugins which says the plugins have to be registered, but I couldn't work out if it's a manual process or if it's done by /usr/share/ipa/wsgi/plugins.py on restart? I'll add the relevant bits to /usr/share/ipa/wsgi/plugins.py for the CLI as well. Should be automatically handled by the plugin.py wsgi handler and related logic in Web UI. Just make sure that the file and the directory have same names (except the extension in file's case of course). 2. Sumit, thanks for the info on Samba, I'll have to leave that now and try it next week. BTW, the version of Samba I'm testing against is 3.6.9-168 on CentOS 6.5. Thanks again for your information and patience, Dylan. On 22 May 2014 14:19, Petr Vobornik pvobo...@redhat.com wrote: On 22.5.2014 14:19, Sumit Bose wrote: On Tue, May 20, 2014 at 02:00:18PM +0100, Dylan Evans wrote: Hello, I need some help with getting Samba and FreeIPA working together. I’ve been following the guide at http://techslaves.org/2011/08/24/freeipa-and-samba-3-integration but that seems quite out of date for IPAv3 and I need some help: yes, it is a bit outdated but still useful. Please note that we are currently working on making the integration of samba more easy. Recently I send a patch to the samba-technical mailing list with a library which would allow samba to use SSSD instead of winbind to look up users and SID-to-name mapping. Alexander is planning to go through the ipasam modules to see how to make integration with Samba file-servers more easy. But coming back to your questions. 1. The guide deals with setting a Samba server SID for one Samba server, but as we have multiple stand-alone Samba3 servers, which SID do I use to create the DNA plugin? Can I enter more than 1 SID? Can I have more than 1 plugin (seems unlikely)? 'net getlocalsid' returns the domain SID and since all you Samba file-servers are member of the IPA domain you can use a common SID here. With IPAv3 SID generation for users and groups is even more easy because you can get it for free by running ipa-adtrust-install (please use the option --add-sids) if you already have users and groups in your IPA server. This prepares the IPA server to be able to create trust relationships to Active Directory and one requirement here is that all users and groups have SID. 'ipa-adtrust-install' will also create a domain SID. 'ipa trustconfig-show' will show the domain SID together with the DNS domain name and the NetBIOS domain name. On your Samba server you should set 'workgroup' to the NetBIOS domain name (see 'net conf list' on the IPA server after running ipa-adtrust-install for a config example). Additionally on your Samba servers you have to set the domain SID in /var/lib/samba/private/secrets.tdb with tdbtool. You will need 3 keys with the same SID SECRETS/SID/DOMNETBIOS - NetBIOS domain name, workgroup in smb.conf SECRETS/SID/DNS.DOMAIN.NAME - DNS domain name, will match realm in smb.conf SECRETS/SID/CLINETBIOS - NetBIOS name of the client, 'netbios name' in smb.conf The SID has to be given in a special binary format. The easiest way to get it is to call 'tdbdump /var/lib/samba/private/secrets.tdb' on the IPA server after running ipa-adtrust-install. The domain SID will always start with \01\04\00\00\00\00\00\05\15\... . You can use this sequence as data for the insert command of tdbtool.
Re: [Freeipa-users] Getting Samba3 and FreeIPAv3 working together
Hello again, Just realised by re-reading this thread that I still needed to create the DNA plugin. I've now done that and I can add users, sorry for being stupid... Dylan. On 3 June 2014 14:44, Dylan Evans devan...@gmail.com wrote: Hi Petr Sumit, I've been trying to get further with my setup. 1. Thanks Petr, the groups.js plugin seems to work fine, it shows the correct info on the GUI screen and seems to be ok. 2. Sumit, I'm afraid that I'm having a few more problems after running ipa-adtrust-install --add-sids. I cannot now add any users on the server (Fedora 20, ipa-server 3.3.5-1) via the command-line or GUI. I get the following error: GUI: IPA Error 4205 missing attribute: sambaSID required by object class sambaSamAccount Command-line: ipa user-add test1234 . ipa: ERROR: missing attribute sambaSID required by object class sambaSamAccount Also, when editing an existing user, there is no sambaSID field available to edit. If you have any ideas, please let me know. Thanks, Dylan. On 26 May 2014 11:40, Petr Vobornik pvobo...@redhat.com wrote: On 23.5.2014 16:31, Dylan Evans wrote: Hi Sumit and Petr, Thanks both of you for your replies, I've now got to go and try to implement all your suggestions but I have some more questions, sorry! The guide at techslaves was fine, I just got stuck with the changes in the JavaScript packages and the Samba server questions. 1. Petr, I put your samba.js plugin into /usr/share/ipa/ui/js/plugins/samba but you'll have to pardon my lack of JS knowledge, anything more than simple Bash scripts tends to leave me confused! Do I need to do anything else apart from restart the IPA service? I read your info at http://pvoborni.fedorapeople.org/doc/#!/guide/Plugins which says the plugins have to be registered, but I couldn't work out if it's a manual process or if it's done by /usr/share/ipa/wsgi/plugins.py on restart? I'll add the relevant bits to /usr/share/ipa/wsgi/plugins.py for the CLI as well. Should be automatically handled by the plugin.py wsgi handler and related logic in Web UI. Just make sure that the file and the directory have same names (except the extension in file's case of course). 2. Sumit, thanks for the info on Samba, I'll have to leave that now and try it next week. BTW, the version of Samba I'm testing against is 3.6.9-168 on CentOS 6.5. Thanks again for your information and patience, Dylan. On 22 May 2014 14:19, Petr Vobornik pvobo...@redhat.com wrote: On 22.5.2014 14:19, Sumit Bose wrote: On Tue, May 20, 2014 at 02:00:18PM +0100, Dylan Evans wrote: Hello, I need some help with getting Samba and FreeIPA working together. I’ve been following the guide at http://techslaves.org/2011/08/24/freeipa-and-samba-3-integration but that seems quite out of date for IPAv3 and I need some help: yes, it is a bit outdated but still useful. Please note that we are currently working on making the integration of samba more easy. Recently I send a patch to the samba-technical mailing list with a library which would allow samba to use SSSD instead of winbind to look up users and SID-to-name mapping. Alexander is planning to go through the ipasam modules to see how to make integration with Samba file-servers more easy. But coming back to your questions. 1. The guide deals with setting a Samba server SID for one Samba server, but as we have multiple stand-alone Samba3 servers, which SID do I use to create the DNA plugin? Can I enter more than 1 SID? Can I have more than 1 plugin (seems unlikely)? 'net getlocalsid' returns the domain SID and since all you Samba file-servers are member of the IPA domain you can use a common SID here. With IPAv3 SID generation for users and groups is even more easy because you can get it for free by running ipa-adtrust-install (please use the option --add-sids) if you already have users and groups in your IPA server. This prepares the IPA server to be able to create trust relationships to Active Directory and one requirement here is that all users and groups have SID. 'ipa-adtrust-install' will also create a domain SID. 'ipa trustconfig-show' will show the domain SID together with the DNS domain name and the NetBIOS domain name. On your Samba server you should set 'workgroup' to the NetBIOS domain name (see 'net conf list' on the IPA server after running ipa-adtrust-install for a config example). Additionally on your Samba servers you have to set the domain SID in /var/lib/samba/private/secrets.tdb with tdbtool. You will need 3 keys with the same SID SECRETS/SID/DOMNETBIOS - NetBIOS domain name, workgroup in smb.conf SECRETS/SID/DNS.DOMAIN.NAME - DNS domain name, will match realm in smb.conf SECRETS/SID/CLINETBIOS - NetBIOS name of the client, 'netbios name' in smb.conf The SID has to be given in a special binary format. The easiest way to get
Re: [Freeipa-users] IPA+AD trust and NFS nobody issue
On 06/03/2014 09:07 AM, Johan Petersson wrote: Hi, Environment: RHEL 7 IPA Server 3.3 with a trust to a Windows 2012 Server AD RHEL 7 NFS Server RHEL 7 Client I have found one problem when using a NFS 4 shared Home Directory for AD users logging in to IPA. I have created a NFS share /home/adexample.org and use autofs map in IPA. All wbinfo tests works as well as id. I can login fine through SSH and Shell with adt...@adexample.org The problem is that I can add the AD user as owner of his Home Directory and if I log in to the NFS Server locally or through ssh permissions are correct but when logging in to any other computer i get nobody as owner. Are those computers RHEL7 NFS clients with SSSD? Can you describe them in more details please? Groups are no problem since AD groups can be mapped to Posix groups. Idmap.conf domain is set to the IPA Domain. Is there some way to get NFS working with the AD user as owner of his Home Directory? Thanks for any help. /This e-mail is private and confidential between the sender and the addressee. / /In the event of misdirection, the recipient is prohibited from using, copying or / /disseminating it or any information in it. Please notify the above if any misdirection./ ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager IdM portfolio Red Hat, Inc. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
