Re: [Freeipa-users] can migrate-ds be safely re-run if it failed...
The groups don't go on the 2nd pass because they already went on the first meant. I meant to reply to this the other day as I have had a lot of experience with re-running migration. Group membership for an already existing group, does NOT come over on the 2nd pass. I have found it is better to start fresh if you want a clean migration. Or, better yet, gather the group memberships via LDAP and migrate them by hand with a friendly script. I through one together to do that pretty easily. ~J On 3/15/16 10:22 AM, Rob Crittenden wrote: lejeczek wrote: On 15/03/16 14:14, lejeczek wrote: On 15/03/16 13:42, Rob Crittenden wrote: lejeczek wrote: On 14/03/16 17:06, Rob Crittenden wrote: lejeczek wrote: with... ipa: ERROR: group LDAP search did not return any result (search base: ou=groups,dc=ccnr,dc=biotechnology, objectclass: groupofuniquenames, groupofnames) I see users went in but later I realized that current samba's ou was "group" not groups. Can I just re-run migrations? Yes. It will skip over anything that already exists in IPA. thanks Rob, may I ask why process by defaults looks up only objectclass: groupofuniquenames, groupofnames? It is conservative but this is why it can be overridden. Is there a reason it skips ldap+samba typical posixGroup & sambaGroupMapping? We haven't had many (any?) reports of migrating from ldap+samba. Lastly, is there a way to preserve account locked/disabled status for posix/samba? I don't know how it is stored but as lon g as the schema is available in IPA then the values should be preserved on migration unless the attributes are associated with a blacklisted objectclass. rob last - this must most FAQ people wonder - can IPA's 389 backend be used in the same/similar fashion samba uses ldap? skipping all the kerberos bits? (samba & IPA on the same one box) this might be more 389-ds related - in old days I remember DS had mozldap dedicated toolset, how is it these days? How do users deal with 389-ds IPA-related bits? many thanks now when I've groups migrated I see mappings user-group are lost. Would it be because my groups did not go in first time together with users? Need more info. What do you mean by mappings are lost? rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Free-IPA failover succeeds, but ssh is broken?
Hi, Try commenting out the proxy command in /etc/ssh/ssh_config The sssd proxy of ssh is buggy as can be. ~J > On Jan 17, 2016, at 05:24, Jakub Hrozekwrote: > > >> On 16 Jan 2016, at 02:21, Jeff Hallyburton >> wrote: >> >> Having finished setting up an ipa server and replica, we're trying to test >> failover to ensure that HA works as expected. We've been able to verify the >> replication agreements and auto-discovery are working, and both servers are >> picked up as expected at install time. >> >> That said, we're seeing some oddities with failover. Once I shut down the >> ipa service on the main ipa server, I get most requests completing after >> about a 2 min window. I am able to: >> >> 1. Authenticate to our jump server and get a kerberos ticket >> 2. kinit successfully as other users >> >> However, whenever I try to ssh to another system within our domain, ssh >> breaks with the following error: >> >> $ ssh -vvv automation01 >> OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013 >> debug1: Reading configuration data /etc/ssh/ssh_config >> debug1: /etc/ssh/ssh_config line 5: Applying options for * >> debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 >> automation01 >> debug1: permanently_drop_suid: 158701 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_rsa type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_rsa-cert type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_dsa type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_dsa-cert type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_ecdsa type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_ecdsa-cert type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_ed25519 type -1 >> debug1: identity file /home/jeff.hallyburton/.ssh/id_ed25519-cert type -1 >> debug1: Enabling compatibility mode for protocol 2.0 >> debug1: Local version string SSH-2.0-OpenSSH_6.6.1 >> ssh_exchange_identification: Connection closed by remote host > > Did you crank up debug level on the machine where sshd is running and see if > anything is logged then? > >> >> Nothing is logged in either /var/log/messages or /var/log/secure when this >> happens, so I'm unsure where to begin debugging. Can you offer any insight? >> >> Thanks, >> >> Jeff >> >> Jeff Hallyburton >> Strategic Systems Engineer >> Bloomip Inc. >> Web: http://www.bloomip.com >> >> Engineering Support: supp...@bloomip.com >> Billing Support: bill...@bloomip.com >> Customer Support Portal: https://my.bloomip.com >> -- >> Manage your subscription for the Freeipa-users mailing list: >> https://www.redhat.com/mailman/listinfo/freeipa-users >> Go to http://freeipa.org for more info on the project > > > -- > Manage your subscription for the Freeipa-users mailing list: > https://www.redhat.com/mailman/listinfo/freeipa-users > Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] tricky one in OpenLDAP migration, groups
Might it be possible with a user-mod or group-add/group-mod to accomplish? Just thinking outside the box I guess. ~J On 1/13/16 7:59 AM, Rob Crittenden wrote: Janelle wrote: Hello, This may not be possible, or if it is I am going to guess it is not going to be easy. If I have an old OpenLDAP environment with users who never had unique UIG/GID - in other words, the GID was not unique to a user, instead it was some global group. Well, I was hoping to migrate over the OpenLDAP domain to IPA, but at the same time create a private group for each user. Just wondering if this might be possible? Example OpenLDAP user=freddy (UID=13) , GID=123456(friday) After migration to IPA: user= uid=13(freddy), gid=13(freddy), groups=123456(friday) Does that make sense? It does but it isn't possible today. In fact the migration won't create user private groups at all (though there is an RFE for that, https://fedorahosted.org/freeipa/ticket/4738 ) I don't think this is an unreasonable request. It may be an extension of the above ticket, probably requiring a new option to deal with the existing primary group. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] tricky one in OpenLDAP migration, groups
Hello, This may not be possible, or if it is I am going to guess it is not going to be easy. If I have an old OpenLDAP environment with users who never had unique UIG/GID - in other words, the GID was not unique to a user, instead it was some global group. Well, I was hoping to migrate over the OpenLDAP domain to IPA, but at the same time create a private group for each user. Just wondering if this might be possible? Example OpenLDAP user=freddy (UID=13) , GID=123456(friday) After migration to IPA: user= uid=13(freddy), gid=13(freddy), groups=123456(friday) Does that make sense? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] tricky one in OpenLDAP migration, groups
Hello, This may not be possible, or if it is I am going to guess it is not going to be easy. If I have an old OpenLDAP environment with users who never had unique UIG/GID - in other words, the GID was not unique to a user, instead it was some global group. Well, I was hoping to migrate over the OpenLDAP domain to IPA, but at the same time create a private group for each user. Just wondering if this might be possible? Example OpenLDAP user=freddy (UID=13) , GID=123456(friday) After migration to IPA: user= uid=13(freddy), gid=13(freddy), groups=123456(friday) Does that make sense? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] 4.2 (or 4.3) clients on 4.1.4 server?
Perfect! Thank you! ~J On 1/12/16 4:24 AM, Martin Kosek wrote: On 01/11/2016 10:21 PM, Janelle wrote: Good day, Just wondering if anyone knows of any reason a 4.2 client running on RHEL 7.2 would have any issues talking to 4.1.4 server on RHEL 7.1? The reason I ask is the process of upgrading. In this case we have to do clients first. If by "talk", you mean use identity, authentication and authorization services - 7.2 talking to 7.1 will work. If by "talk" you mean "ipa" management tool, then this would not work unless special option is used (see recent thread from Jan Pazdziora for details). Details: http://www.freeipa.org/page/Client#Compatibility Martin -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] 4.2 (or 4.3) clients on 4.1.4 server?
Good day, Just wondering if anyone knows of any reason a 4.2 client running on RHEL 7.2 would have any issues talking to 4.1.4 server on RHEL 7.1? The reason I ask is the process of upgrading. In this case we have to do clients first. Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] SSSD to IPA connection?
Happy New Year everyone! I came across a couple of my servers having some strange connection problems and was wondering if anyone else has seen this or know what might cause it? This is IPA 4.1.4 and client on RHEL 7.1. When you look at the status, for some reason, SSSD has lost contact with the servers, and a restart is required. What I don't understand is what this "Preauth" failure is? Ideas? ~Janelle Redirecting to /bin/systemctl status sssd.service sssd.service - System Security Services Daemon Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled) Drop-In: /etc/systemd/system/sssd.service.d └─journal.conf Active: active (running) since Sat 2015-12-12 07:41:55 EST; 2 weeks 4 days ago Process: 24482 ExecStart=/usr/sbin/sssd -D -f (code=exited, status=0/SUCCESS) Main PID: 24483 (sssd) CGroup: /system.slice/sssd.service ├─24483 /usr/sbin/sssd -D -f ├─24484 /usr/libexec/sssd/sssd_be --domain example.com --uid 0 --gid 0 --debug-to-files ├─24485 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files ├─24486 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --debug-to-files ├─24487 /usr/libexec/sssd/sssd_ssh --uid 0 --gid 0 --debug-to-files └─24488 /usr/libexec/sssd/sssd_pac --uid 0 --gid 0 --debug-to-files Jan 01 07:55:24 client.example.com [sssd[krb5_child[10456]]][10456]: Preauthentication failed Jan 01 07:56:07 client.example.com [sssd[krb5_child[10464]]][10464]: Preauthentication failed Jan 01 07:57:16 client.example.com [sssd[krb5_child[10471]]][10471]: Preauthentication failed Jan 01 08:10:48 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:48 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:49 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:49 client.example.com sssd_be[12345]: GSSAPI client step 2 Jan 01 08:20:10 client.example.com [sssd[krb5_child[10538]]][10538]: Preauthentication failed Jan 01 08:20:29 client.example.com [sssd[krb5_child[10541]]][10541]: Preauthentication failed Jan 01 08:20:48 client.example.com [sssd[krb5_child[10596]]][10596]: Preauthentication failed -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] SSSD to IPA connection?
When this happens - it stops accepting logins for any of my users. I have to restart SSSD to get it to work again. And it is just kind of random when this happens. How can a STATUS command sent to SSSD show a wrong password? ~J On 1/4/16 9:11 AM, Jakub Hrozek wrote: On Mon, Jan 04, 2016 at 08:30:08AM -0800, Janelle wrote: Happy New Year everyone! I came across a couple of my servers having some strange connection problems and was wondering if anyone else has seen this or know what might cause it? This is IPA 4.1.4 and client on RHEL 7.1. When you look at the status, for some reason, SSSD has lost contact with the servers, and a restart is required. What I don't understand is what this "Preauth" failure is? Ideas? ~Janelle Redirecting to /bin/systemctl status sssd.service sssd.service - System Security Services Daemon Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled) Drop-In: /etc/systemd/system/sssd.service.d └─journal.conf Active: active (running) since Sat 2015-12-12 07:41:55 EST; 2 weeks 4 days ago Process: 24482 ExecStart=/usr/sbin/sssd -D -f (code=exited, status=0/SUCCESS) Main PID: 24483 (sssd) CGroup: /system.slice/sssd.service ├─24483 /usr/sbin/sssd -D -f ├─24484 /usr/libexec/sssd/sssd_be --domain example.com --uid 0 --gid 0 --debug-to-files ├─24485 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files ├─24486 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --debug-to-files ├─24487 /usr/libexec/sssd/sssd_ssh --uid 0 --gid 0 --debug-to-files └─24488 /usr/libexec/sssd/sssd_pac --uid 0 --gid 0 --debug-to-files Jan 01 07:55:24 client.example.com [sssd[krb5_child[10456]]][10456]: Preauthentication failed Jan 01 07:56:07 client.example.com [sssd[krb5_child[10464]]][10464]: Preauthentication failed Jan 01 07:57:16 client.example.com [sssd[krb5_child[10471]]][10471]: Preauthentication failed Preauthentication failed means more or less wrong password, but since the message is from krb5_child, I guess it's during user login. What exactly is not working? Jan 01 08:10:48 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:48 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:49 client.example.com sssd_be[12345]: GSSAPI client step 1 Jan 01 08:10:49 client.example.com sssd_be[12345]: GSSAPI client step 2 Jan 01 08:20:10 client.example.com [sssd[krb5_child[10538]]][10538]: Preauthentication failed Jan 01 08:20:29 client.example.com [sssd[krb5_child[10541]]][10541]: Preauthentication failed Jan 01 08:20:48 client.example.com [sssd[krb5_child[10596]]][10596]: Preauthentication failed -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] otpd heavy load?
I'll gather up the info first chance I get. Thank you ~J On 12/14/15 7:35 AM, Alexander Bokovoy wrote: On Thu, 10 Dec 2015, Janelle wrote: libverto-tevent-0.2.5-4.el7.x86_64 libverto-0.2.5-4.el7.x86_64 Patching problem perhaps? Can you install debuginfo for krb5 and ipa? And then install ltrace? I would go with these tools: - once ipa-otpd recreates its high resource usage, run 'pstack ' periodically to take few snapshots of its stacktraces - do 'ltrace -s 256 -S -ttt -T -r -p ' Save the reports you'd get, and make them available to us. They will be big enough, so avoid sending it to the list, send privately. On 12/10/15 10:49 AM, Nathaniel McCallum wrote: Please provide the output of the 'rpm -qa libverto*' command. On Thu, 2015-12-10 at 10:13 -0800, Janelle wrote: RHEL 7.1 On 12/10/15 9:55 AM, Nathaniel McCallum wrote: On Thu, 2015-12-10 at 09:34 -0800, Janelle wrote: Hi, Hope everyone is enjoying the holiday season. Been away for sometime, and wanted to jump in with a new question. I am seeing otpd have high resource usage (from just monitoring via top, sar and uptime) however, I can not seem to find any logging from it, nor how I might be able to enable some in order to find out why it is using so much CPU? Any thoughts/suggestions? Log messages should be available through journalctl. Which libverto backend are you running? If you are on Fedora, please provide the output of the 'rpm -qa libverto*' command. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] otpd heavy load?
libverto-tevent-0.2.5-4.el7.x86_64 libverto-0.2.5-4.el7.x86_64 Patching problem perhaps? On 12/10/15 10:49 AM, Nathaniel McCallum wrote: Please provide the output of the 'rpm -qa libverto*' command. On Thu, 2015-12-10 at 10:13 -0800, Janelle wrote: RHEL 7.1 On 12/10/15 9:55 AM, Nathaniel McCallum wrote: On Thu, 2015-12-10 at 09:34 -0800, Janelle wrote: Hi, Hope everyone is enjoying the holiday season. Been away for sometime, and wanted to jump in with a new question. I am seeing otpd have high resource usage (from just monitoring via top, sar and uptime) however, I can not seem to find any logging from it, nor how I might be able to enable some in order to find out why it is using so much CPU? Any thoughts/suggestions? Log messages should be available through journalctl. Which libverto backend are you running? If you are on Fedora, please provide the output of the 'rpm -qa libverto*' command. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] otpd heavy load?
RHEL 7.1 On 12/10/15 9:55 AM, Nathaniel McCallum wrote: On Thu, 2015-12-10 at 09:34 -0800, Janelle wrote: Hi, Hope everyone is enjoying the holiday season. Been away for sometime, and wanted to jump in with a new question. I am seeing otpd have high resource usage (from just monitoring via top, sar and uptime) however, I can not seem to find any logging from it, nor how I might be able to enable some in order to find out why it is using so much CPU? Any thoughts/suggestions? Log messages should be available through journalctl. Which libverto backend are you running? If you are on Fedora, please provide the output of the 'rpm -qa libverto*' command. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] otpd heavy load?
Hi, Hope everyone is enjoying the holiday season. Been away for sometime, and wanted to jump in with a new question. I am seeing otpd have high resource usage (from just monitoring via top, sar and uptime) however, I can not seem to find any logging from it, nor how I might be able to enable some in order to find out why it is using so much CPU? Any thoughts/suggestions? Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] OTP vs password?
Hello all... Seeing something very strange. With OTP enabled for all users - here is the configuration: Some hosts fully "enrolled" with IPA, and some are simply configured with authconfig to use LDAP backend for authentication. RANDOMLY < Keyword here -- all systems use SSSD regardless of the authentication method. A user will be able to login with password+token, but the random part - sometimes JUST the password. Is this possible due to some odd caching issues with SSSD perhaps or ??? How might I research this? is there anything to look for in configs or logs? thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] clean-ruv : How Long?
Hello, I was wondering if there is any average or expectation of how long a "clean-ruv" task should take across 16 fairly busy servers? Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] unindexed searches?
Hello, I hope this is a simply question. I have 1000's of these on my servers and it severely bogs them down. Any ideas on how to get rid of unindexed searches? [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11158 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11160 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11163 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11166 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11168 RESULT err=0 tag=101 nentries=0 etime=0 notes=U [04/Oct/2015:13:27:54 -0700] conn=1344502 op=11171 RESULT err=0 tag=101 nentries=0 etime=0 notes=U ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] More replication fun
On 10/5/15 10:16 AM, Simo Sorce wrote: On 05/10/15 11:11, Janelle wrote: So here is a fun question -- how is this possible? from ipa-replica-manage list-ruv ipa002.example.com 389 6 ipa003.example.com 389 30 <- Huh??? ipa003.example.com 389 33 <- ipa004.example.com 389 24 ipa003 was reinstalled but the RUV was not properly cleaned Simo. So there is no conflict even with a duplicate like that? I mean the server only physically exists once, so I guess it should not cause a problem. I mean replication seems to be working, it just seems odd that I saw that. You would think that the normal ipa-replica-manage "del" options would do all the proper cleaning, but apparently not. Thanks for the clarification. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] admin loses access?
On 10/5/15 7:39 AM, Rob Crittenden wrote: Torsten Harenberg wrote: Hi Janelle, Am 04.10.2015 um 19:25 schrieb Janelle: Just wondering if anyone knows why this happens from time to time on servers: $ kinit admin kinit: Clients credentials have been revoked while getting initial credentials there are no failed logins to the admin account - not even any login attempts, so it is not like someone is getting the account locked out. Just curious if anyone else has seen in. With 16 masters, it occurs randomly on some hosts, but eventually clears either on its own or with a restart of IPA. However, I just restarted IPA on this server and after about 2-3 minutes it works again. I am seeing the same, see my mail "kinit admin not working anymore (LOCKED_OUT: Clients credentials have been revoked)" from 03-SEPT. Actually, wasn't it you who wanted to open a ticket? Have a nice evening, Torsten When you see this run `ipa user-status admin` and `ipa pwpolicy-show --user=admin` and provide that so we can potentially see what is going on. rob I am curious -- if you have 16 masters, but this only happens once in awhile on 1 or 2 servers, how does the pwpolicy fit in? I am trying to understand the thinking of where you are going?? I will for sure do this the next time it happens, but I want to understand logic? thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] More replication fun
So here is a fun question -- how is this possible? from ipa-replica-manage list-ruv ipa002.example.com 389 6 ipa003.example.com 389 30 <- Huh??? ipa003.example.com 389 33 <- ipa004.example.com 389 24 ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] admin loses access?
Hello everyone, Just wondering if anyone knows why this happens from time to time on servers: $ kinit admin kinit: Clients credentials have been revoked while getting initial credentials there are no failed logins to the admin account - not even any login attempts, so it is not like someone is getting the account locked out. Just curious if anyone else has seen in. With 16 masters, it occurs randomly on some hosts, but eventually clears either on its own or with a restart of IPA. However, I just restarted IPA on this server and after about 2-3 minutes it works again. Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] password resets - errors
On 9/28/15 11:33 AM, Rob Crittenden wrote: Simo Sorce wrote: On 27/09/15 09:21, Janelle wrote: Hello, I continue to see these a lot, but only on some servers. It causes a lot of confusions with my users. There must be a way to troubleshoot this and find the issue. Also, there is nothing wrong with the password policies. They are all set to default, and this occurs even when a user's password has expired. The only thing I can say is it tends to happen on more heavily loaded servers than lightly loaded ones. And perhaps the most important point - the password *IS* changed successfully! Changing password for user expired-user. Current Password: New password: Retype new password: Password change failed. Server message: Current password's minimum life has not expired Password not changed. passwd: Authentication token manipulation error Thoughts? Anything? This may be due to an implementation issue in the client. libkrb5 tends to wait only 1 second for an operation to succeed/fail and will send a new (identical) message if it gets back no answer, this is due to the fact historically KRB5 has used UDP in preference which doesn't guarantee message delivery, so the only option is to retry. However if the first message actually went through and the only problem is that the server was busy and slower a second message will be received and processed just the same, only to find out the password has just been changed and can't be changed again, hence the error message. I guess one way to handle this would be to disable clients from using UDP completely, although I am not 100% certain this will avoid the problem, IIRC at least in some versions the client library would retry after 1 second even on TCP. Simo. udp_preference_limit 0 was added to /etc/krb5.conf in 4.2 to prefer TCP for the initial request anyway. According to the man page it will always fall back to UDP upon failure. rob Something to add: If I get the failure and then try again, I get: Current Password: System is offline, password change not possible passwd: Authentication token manipulation error I also tried setting the kdc_timeout to 10 seconds (it says the default is 3) tcpdump shows everything running over TCP and it never goes to UDP at all, so this is now out of the picture. Any other ideas? I guess I need to turn on debug_level to something about 7 to try and figure this out. ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] password resets - errors
On 9/28/15 11:33 AM, Rob Crittenden wrote: Simo Sorce wrote: On 27/09/15 09:21, Janelle wrote: Hello, I continue to see these a lot, but only on some servers. It causes a lot of confusions with my users. There must be a way to troubleshoot this and find the issue. Also, there is nothing wrong with the password policies. They are all set to default, and this occurs even when a user's password has expired. The only thing I can say is it tends to happen on more heavily loaded servers than lightly loaded ones. And perhaps the most important point - the password *IS* changed successfully! Changing password for user expired-user. Current Password: New password: Retype new password: Password change failed. Server message: Current password's minimum life has not expired Password not changed. passwd: Authentication token manipulation error Thoughts? Anything? This may be due to an implementation issue in the client. libkrb5 tends to wait only 1 second for an operation to succeed/fail and will send a new (identical) message if it gets back no answer, this is due to the fact historically KRB5 has used UDP in preference which doesn't guarantee message delivery, so the only option is to retry. However if the first message actually went through and the only problem is that the server was busy and slower a second message will be received and processed just the same, only to find out the password has just been changed and can't be changed again, hence the error message. I guess one way to handle this would be to disable clients from using UDP completely, although I am not 100% certain this will avoid the problem, IIRC at least in some versions the client library would retry after 1 second even on TCP. Simo. udp_preference_limit 0 was added to /etc/krb5.conf in 4.2 to prefer TCP for the initial request anyway. According to the man page it will always fall back to UDP upon failure. rob This value appears to be set in 4.1.x as well, at least it is on my configurations. Policy is set: Group: global_policy Max lifetime (days): 90 Min lifetime (hours): 1 and this is true for ALL users. I will try disabling UDP completely. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] password resets - errors
On 9/28/15 6:10 AM, Rob Crittenden wrote: Janelle wrote: Hello, I continue to see these a lot, but only on some servers. It causes a lot of confusions with my users. There must be a way to troubleshoot this and find the issue. Also, there is nothing wrong with the password policies. They are all set to default, and this occurs even when a user's password has expired. The only thing I can say is it tends to happen on more heavily loaded servers than lightly loaded ones. And perhaps the most important point - the password *IS* changed successfully! Changing password for user expired-user. Current Password: New password: Retype new password: Password change failed. Server message: Current password's minimum life has not expired Password not changed. passwd: Authentication token manipulation error Thoughts? Anything? ~Janelle What tool is changing the expired password? I'd be curious to see the password policy for the user, ipa pwpolicy-show --user= Seeing the krbLastPwdChange and krbPasswordExpiration might be handy too. rob Hi, I was hoping it would not go off on this tangent. All users have the default PW policy -- there are no differences and every single user has the same problem. The tool is simple "passwd" or, in the case of some users who have actually hit the 90 expiry, nothing more than a simple login followed by the system saying your password has expired, please change it. The krbLastPwdChange shows the exact day/time of the user changing their PW, in this case, when this error occurs. The expiration shows 90 days from that time. If you see the specifics I mentioned, even though the error is presented, the password is actually changed. Really confused with this one. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] password resets - errors
On 9/28/15 6:10 AM, Rob Crittenden wrote: Janelle wrote: Hello, I continue to see these a lot, but only on some servers. It causes a lot of confusions with my users. There must be a way to troubleshoot this and find the issue. Also, there is nothing wrong with the password policies. They are all set to default, and this occurs even when a user's password has expired. The only thing I can say is it tends to happen on more heavily loaded servers than lightly loaded ones. And perhaps the most important point - the password *IS* changed successfully! Changing password for user expired-user. Current Password: New password: Retype new password: Password change failed. Server message: Current password's minimum life has not expired Password not changed. passwd: Authentication token manipulation error Thoughts? Anything? ~Janelle What tool is changing the expired password? I'd be curious to see the password policy for the user, ipa pwpolicy-show --user= Seeing the krbLastPwdChange and krbPasswordExpiration might be handy too. rob And, please accept my apology if that was worded poorly on my reply. Very appreciative for the help, just was trying to steer away from the actual password policy having anything to do with it. As I re-read my reply, I thought it might have sounded rude in the email. Not intended to be that way. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] password resets - errors
Hello, I continue to see these a lot, but only on some servers. It causes a lot of confusions with my users. There must be a way to troubleshoot this and find the issue. Also, there is nothing wrong with the password policies. They are all set to default, and this occurs even when a user's password has expired. The only thing I can say is it tends to happen on more heavily loaded servers than lightly loaded ones. And perhaps the most important point - the password *IS* changed successfully! Changing password for user expired-user. Current Password: New password: Retype new password: Password change failed. Server message: Current password's minimum life has not expired Password not changed. passwd: Authentication token manipulation error Thoughts? Anything? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] V6 and v4
On 9/24/15 12:57 AM, Martin Kosek wrote: On 09/23/2015 10:05 PM, Janelle wrote: On 9/13/15 11:46 PM, Alexander Bokovoy wrote: On Sun, 13 Sep 2015, Janelle wrote: Hello, I read something recently that if ip v6 is disable on a server this hurts performance in some way? Is there more info on this or did I misread it? Do not disable IPv6 stack on your machines. By disabling IPv6 you are not doing good. On contrary, many contemporary software projects are using IPv6-enabled network calls by default because both IPv6 and IPv4 share the same name space on the machine so you only need to listen on a IPv6 port to accept both IPv4 and IPv6. This is a recommended approach for networking applications' developers for years already. Note that this means only that support for IPv6 stack is enabled in the kernel. You are not required to go with IPv6 networking addresses, this is not really needed if you don't want to. But allowing applications to be IPv6 aware is required. FreeIPA has several components which are programmed in such way that they expect IPv6 stack to be enabled for reasons outlined above. If you disable IPv6 stack, FreeIPA will partially malfunction and will not really be in a supported state, especially when we are talking about trusts to Active Directory (and, in future, IPA to IPA trust). BTW - I did re-enable IPv6 and was able to "clean ruv" all the "dead" entries, which I had not been able to do before. Thank you for this. Hello Janelle, Thanks for confirmation! I added this knowledge to http://www.freeipa.org/page/Troubleshooting#Obsolete_RUV_records as it is definitely not an obvious fix to resolve the RUV issue. Please feel very welcome to extend Troubleshooting guide if you have other advise that could help others speed up their RUV investigation - you have definitely a lot of experience with them. Thanks! Martin Final - Final confirmation now. I now deleted a replica and re-added. No "ghost" entries at all. Everything is perfect. Yeah, this was crazy that it was the fix on all the problems I had for a few months. It definitely was not an obvious one. I had wondered if it was DNS at one point, but every server/master has a /etc/hosts file with all hostnames and IPs (I never trust DNS). Thank you for sticking with all my issues and helping with this. This one was a huge help. At one point I had 9 of these ghost RUVs that would not go away. Even if I deleted them off a server, they would magically re-appear. It was so frustrating. Having a clean environment is a wonderful thing. I love IPA!! I will check the DOCs and if there is anything I can add I will. ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Ghost user?
I have a user I created for testing, but now shows as both "there" but not there.. *ipa user-show jtest* ipa: ERROR: jtest: user not found *ipa user-find jtest* -- 1 user matched -- User login: jtest First name: janelle Last name: test Home directory: /home/jtest Login shell: /bin/bash Email address: jt...@example.com UID: 1372025403 GID: 1372025403 Account disabled: False Password: True Kerberos keys available: True *ipa user-del jtest* ipa: ERROR: jtest: user not found *ipa user-add jtest* First name: janelle Last name: jtest ipa: ERROR: user with name "jtest" already exists I am officially baffled. Any ideas? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Ghost user?
On 9/23/15 10:36 AM, Martin Basti wrote: On 09/23/2015 07:15 PM, Janelle wrote: I have a user I created for testing, but now shows as both "there" but not there.. *ipa user-show jtest* ipa: ERROR: jtest: user not found *ipa user-find jtest* -- 1 user matched -- User login: jtest First name: janelle Last name: test Home directory: /home/jtest Login shell: /bin/bash Email address: jt...@example.com UID: 1372025403 GID: 1372025403 Account disabled: False Password: True Kerberos keys available: True *ipa user-del jtest* ipa: ERROR: jtest: user not found *ipa user-add jtest* First name: janelle Last name: jtest ipa: ERROR: user with name "jtest" already exists I am officially baffled. Any ideas? ~Janelle Hello, can you please check directly with LDAP search, if it is replication conflict? Martin I am not sure what I am looking for to determine the replication conflict you speak of. I know how to use ldapsearch, but not sure what you want to see. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] V6 and v4
On 9/13/15 11:46 PM, Alexander Bokovoy wrote: On Sun, 13 Sep 2015, Janelle wrote: Hello, I read something recently that if ip v6 is disable on a server this hurts performance in some way? Is there more info on this or did I misread it? Do not disable IPv6 stack on your machines. By disabling IPv6 you are not doing good. On contrary, many contemporary software projects are using IPv6-enabled network calls by default because both IPv6 and IPv4 share the same name space on the machine so you only need to listen on a IPv6 port to accept both IPv4 and IPv6. This is a recommended approach for networking applications' developers for years already. Note that this means only that support for IPv6 stack is enabled in the kernel. You are not required to go with IPv6 networking addresses, this is not really needed if you don't want to. But allowing applications to be IPv6 aware is required. FreeIPA has several components which are programmed in such way that they expect IPv6 stack to be enabled for reasons outlined above. If you disable IPv6 stack, FreeIPA will partially malfunction and will not really be in a supported state, especially when we are talking about trusts to Active Directory (and, in future, IPA to IPA trust). BTW - I did re-enable IPv6 and was able to "clean ruv" all the "dead" entries, which I had not been able to do before. Thank you for this. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] V6 and v4
On 9/13/15 11:46 PM, Alexander Bokovoy wrote: On Sun, 13 Sep 2015, Janelle wrote: Hello, I read something recently that if ip v6 is disable on a server this hurts performance in some way? Is there more info on this or did I misread it? Do not disable IPv6 stack on your machines. By disabling IPv6 you are not doing good. On contrary, many contemporary software projects are using IPv6-enabled network calls by default because both IPv6 and IPv4 share the same name space on the machine so you only need to listen on a IPv6 port to accept both IPv4 and IPv6. This is a recommended approach for networking applications' developers for years already. Note that this means only that support for IPv6 stack is enabled in the kernel. You are not required to go with IPv6 networking addresses, this is not really needed if you don't want to. But allowing applications to be IPv6 aware is required. FreeIPA has several components which are programmed in such way that they expect IPv6 stack to be enabled for reasons outlined above. If you disable IPv6 stack, FreeIPA will partially malfunction and will not really be in a supported state, especially when we are talking about trusts to Active Directory (and, in future, IPA to IPA trust). Now it makes me wonder if my problems with replicas and RUVs were caused by v6 being disabled. Time for some investigation. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] 4.1 -> 4.2
Here is an interesting problem. Currently running 4.1 on RHEL 7.1 -- I would like to migrate to 4.2, but that seems to only be running on Fedora these days. Is there a way to bring up a 4.2.1c and migrate to it from 4.1c using the ipa migrate tool? Or is theree another way possible?? thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] 4.1 -> 4.2
thank you - just downloaded the beta to check it out. ~J On 9/17/15 10:20 AM, Alexander Bokovoy wrote: On Thu, 17 Sep 2015, Janelle wrote: Here is an interesting problem. Currently running 4.1 on RHEL 7.1 -- I would like to migrate to 4.2, but that seems to only be running on Fedora these days. Is there a way to bring up a 4.2.1c and migrate to it from 4.1c using the ipa migrate tool? Or is theree another way possible?? Just wait for RHEL 7.2 release. Beta version is already out and it includes IPA 4.2. http://red.ht/1i65UND -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] V6 and v4
On 9/13/15 11:46 PM, Alexander Bokovoy wrote: On Sun, 13 Sep 2015, Janelle wrote: Hello, I read something recently that if ip v6 is disable on a server this hurts performance in some way? Is there more info on this or did I misread it? Do not disable IPv6 stack on your machines. By disabling IPv6 you are not doing good. On contrary, many contemporary software projects are using IPv6-enabled network calls by default because both IPv6 and IPv4 share the same name space on the machine so you only need to listen on a IPv6 port to accept both IPv4 and IPv6. This is a recommended approach for networking applications' developers for years already. Note that this means only that support for IPv6 stack is enabled in the kernel. You are not required to go with IPv6 networking addresses, this is not really needed if you don't want to. But allowing applications to be IPv6 aware is required. FreeIPA has several components which are programmed in such way that they expect IPv6 stack to be enabled for reasons outlined above. If you disable IPv6 stack, FreeIPA will partially malfunction and will not really be in a supported state, especially when we are talking about trusts to Active Directory (and, in future, IPA to IPA trust). Currently no AD trusts and none planned ever, but based on your suggestions, I will re-enable the v6 stack. thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] V6 and v4
Hello, I read something recently that if ip v6 is disable on a server this hurts performance in some way? Is there more info on this or did I misread it? Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Logging?
On 9/11/15 3:25 AM, Jakub Hrozek wrote: On Thu, Sep 10, 2015 at 08:05:16AM -0700, Janelle wrote: On 9/10/15 7:55 AM, Martin Kosek wrote: On 09/09/2015 09:50 PM, Janelle wrote: Hello, I was wondering if anyone has played with thee extended logging of IPA and specifically SSSD and the kibana dashboards they put together. https://www.freeipa.org/page/Centralized_Logging I can't seem to get "clients" to send the login info (https://www.freeipa.org/images/6/65/Rek-user-logins.png) , even though I see the data in the logs, and was wondering if anyone has any tips? Thank you ~Janelle Thanks for feedback, I am CCing Peter Schiffer and Jakub Hrozek who were involved more in the client parts. What did you run for configuring the client? ipa-log-config from https://github.com/pschiffe/ipa-log-config ? Hi Martin, Yes, I did run the log config tool. It works flawlessly on the IPA servers, but although it claims it sets everything up on clients, I am seeing no actual data, even though, there is data in the logs themselves.. So I am busy trying to debug where rsyslog is missing something. I am more of a syslog-ng person, so I am having to learn all the bits and pieces of rsyslog, and perhaps I am missing something. To further help -- I have tried 2 methods of a client. One with a client that was "enrolled" via standard ipa-client-install, and another LDAP-only client, still using SSSD but only configured with LDAP settings for Auth. I would suggest to debug step by step -- are the sssd debug logs being generated? Are they being collected by rsyslog? etc.. Ok. Thank you. I had stated that the logs are indeed being populated correctly. I guess it is something with the rsyslog config being set by the tool. I will try and debug that. The odd thing is, the settings are the same on the IPA server, and it logs correct, but not on clients. Oh well, back to the drawing board. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Logging?
On 9/10/15 7:55 AM, Martin Kosek wrote: On 09/09/2015 09:50 PM, Janelle wrote: Hello, I was wondering if anyone has played with thee extended logging of IPA and specifically SSSD and the kibana dashboards they put together. https://www.freeipa.org/page/Centralized_Logging I can't seem to get "clients" to send the login info (https://www.freeipa.org/images/6/65/Rek-user-logins.png) , even though I see the data in the logs, and was wondering if anyone has any tips? Thank you ~Janelle Thanks for feedback, I am CCing Peter Schiffer and Jakub Hrozek who were involved more in the client parts. What did you run for configuring the client? ipa-log-config from https://github.com/pschiffe/ipa-log-config ? Hi Martin, Yes, I did run the log config tool. It works flawlessly on the IPA servers, but although it claims it sets everything up on clients, I am seeing no actual data, even though, there is data in the logs themselves.. So I am busy trying to debug where rsyslog is missing something. I am more of a syslog-ng person, so I am having to learn all the bits and pieces of rsyslog, and perhaps I am missing something. To further help -- I have tried 2 methods of a client. One with a client that was "enrolled" via standard ipa-client-install, and another LDAP-only client, still using SSSD but only configured with LDAP settings for Auth. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Logging?
Hello, I was wondering if anyone has played with thee extended logging of IPA and specifically SSSD and the kibana dashboards they put together. https://www.freeipa.org/page/Centralized_Logging I can't seem to get "clients" to send the login info (https://www.freeipa.org/images/6/65/Rek-user-logins.png) , even though I see the data in the logs, and was wondering if anyone has any tips? Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] kinit admin not working anymore (LOCKED_OUT: Clients credentials have been revoked)
Sorry Rob - I beg to differ here. I can replicate this with my replica failures. It happens that a replica simply loses it's mind. Somehow the keytab gets mucked up and further connections for replication fail -- it shows a failed "admin" login and they add up because the other servers continue. It only happens on the failed replica -- AND I am the only one with the admin PW and there are ZERO failed attempts over SSH. As soon as I get another failed replica in this state (about once every 2-3 weeks) I will post the logs and open a ticket. On one server, I simply did a reboot, and when it came back, the keytab was wrong and the replica now claimed that it was no longer a member of the replica list. Let me get more information and logs to open a ticket. ~J On 9/3/15 11:11 AM, Rob Crittenden wrote: Janelle wrote: You will find, if you check in the ns-slapd "errors" log that this server may no longer be handling replication correctly. Look in /var/log/dirsrv/slapd-INSTANCE/errors This probably doesn't have anything to do with replication. Lockout is per-master because failed (and successful) logins are not replicated due to the performance issues that would bring. Image 500 people all logging in at the same time in the morning how busy all the masters would be replicating the successes and failures. So this is a perfectly reasonable scenario where on one master the admin has violated the password lockout policy and is locked out but can still log in to other masters. ipa user-status will show the lockout attributes by master. And ipa user-unlock will unlock them. rob Look for errors where replication is not starting correctly because of credential problems. You may have to re-init this replica. The reason "admin" is locked out is that something gets screwed up with the keytab file that was originally installed (I have not found the cause yet, only experienced the exact same thing) Once the keytab file is messed up, others servers can't authenticate and therefore the ADMIN account gets locked out. If you restart the server, it will clear for a little while, but go rgiht back to being locked out. Solution - delete the replica and recreate. ~J On 9/3/15 2:08 AM, Torsten Harenberg wrote: Dear all, I cannot get an "admin" kerberos token anymore on our main IPA server: [root@ipa log]# kinit admin kinit: Clients credentials have been revoked while getting initial credentials Sep 03 11:02:30 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: LOCKED_OUT: ad...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Clients credentials have been revoked also login via HTTP is not possible anymore: Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: NEEDED_PREAUTH: HTTP/ipa.pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Additional pre-authentication required Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): closing down fd 11 Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: ISSUE: authtime 1441271092, etypes {rep=18 tkt=18 ses=18}, HTTP/ipa.pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): closing down fd 11 Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: LOCKED_OUT: ad...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Clients credentials have been revoked while the same works on the secondary server. I read http://web.mit.edu/kerberos/krb5-devel/doc/admin/lockout.html but this did not give me a clue how to get out of this. I am pretty sure that I never entered a wrong password, but of course someone could have tried to log in on the Web interface. Any idea how this can be resolved? Kind regards Torsten -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] kinit admin not working anymore (LOCKED_OUT: Clients credentials have been revoked)
You will find, if you check in the ns-slapd "errors" log that this server may no longer be handling replication correctly. Look in /var/log/dirsrv/slapd-INSTANCE/errors Look for errors where replication is not starting correctly because of credential problems. You may have to re-init this replica. The reason "admin" is locked out is that something gets screwed up with the keytab file that was originally installed (I have not found the cause yet, only experienced the exact same thing) Once the keytab file is messed up, others servers can't authenticate and therefore the ADMIN account gets locked out. If you restart the server, it will clear for a little while, but go rgiht back to being locked out. Solution - delete the replica and recreate. ~J On 9/3/15 2:08 AM, Torsten Harenberg wrote: Dear all, I cannot get an "admin" kerberos token anymore on our main IPA server: [root@ipa log]# kinit admin kinit: Clients credentials have been revoked while getting initial credentials Sep 03 11:02:30 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: LOCKED_OUT: ad...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Clients credentials have been revoked also login via HTTP is not possible anymore: Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: NEEDED_PREAUTH: HTTP/ipa.pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Additional pre-authentication required Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): closing down fd 11 Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: ISSUE: authtime 1441271092, etypes {rep=18 tkt=18 ses=18}, HTTP/ipa.pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): closing down fd 11 Sep 03 11:04:52 ipa.pleiades.uni-wuppertal.de krb5kdc[1351](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 132.195.124.12: LOCKED_OUT: ad...@pleiades.uni-wuppertal.de for krbtgt/pleiades.uni-wuppertal...@pleiades.uni-wuppertal.de, Clients credentials have been revoked while the same works on the secondary server. I read http://web.mit.edu/kerberos/krb5-devel/doc/admin/lockout.html but this did not give me a clue how to get out of this. I am pretty sure that I never entered a wrong password, but of course someone could have tried to log in on the Web interface. Any idea how this can be resolved? Kind regards Torsten -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Ipa add-user non interactively specifying a password.
You could use --random instead of --password, which will force a nice 10 char random PW that can be captured and sent to your user. ~J On 9/1/15 12:54 PM, Chris Mohler wrote: Thanks Craig! That's quite a handy reply. It's actually a lot nicer than what I was planning to do. I appreciate this a lot. -Chris On 09/01/2015 03:33 PM, Craig White wrote: -Original Message- From: freeipa-users-boun...@redhat.com [mailto:freeipa-users-boun...@redhat.com] On Behalf Of Chris Mohler Sent: Tuesday, September 01, 2015 12:17 PM To: freeipa-users@redhat.com Subject: [Freeipa-users] Ipa add-user non interactively specifying a password. Hi List, I'm trying to make a script to add users non interactively with ipa add-user and specify a password of testpw I tried: ipa user-add username --first=firstname --last=lastname --homedir=/home/username --password testpw --gidnumber= --noprivate --shell=/bin/bash #ipa: ERROR: command 'user_add' takes at most 1 argument and this: ipa user-add username --first=firstname --last=lastname --homedir=/home/username --password=testpw --gidnumber= --noprivate --shell=/bin/bash #ipa: error: --password option does not take a value No Luck. Any suggestions? - I will take it a lot further - salt to taste (and watch the line wraps)... #!/bin/sh # # Script to automate adding users # # Updated 12/16/2014 # Craig White # CMD1='/usr/bin/ipa user-add' CMD2='/usr/bin/ipa group-add-member' TEE='/usr/bin/tee -a' LOG='/tmp/ipa_users_add.txt' MAIL='/bin/mailx' KERB=`klist -s; echo $?` $LOG [[ -n "$4" ]] || { echo "Usage: ipa_user_add.sh LOGIN FIRST_NAME LAST_NAME EMAIL GROUPS " && echo " REQUIRED > ^ ^ ^ ^" && echo "You can have many groups separated with just a space"; exit 0 ; } [[ $KERB == "0" ]] || { echo "Your kerberos ticket has expired - Please create a valid kerberos ticket by typing 'kinit'"; exit 0 ; } if [ -z "$EMAIL" ]; then echo "You need to add EMAIL to your environment variables - type 'export EMAIL=YOUR_EMAIL_ADDRESS' before running this command or better yet, add it to your .bash_profile" exit 0 fi $CMD1 $1 --first=$2 --last=$3 --random --email=$4 | $TEE $LOG echo " - - - -" | $TEE $LOG echo "You must login and change your password" | $TEE $LOG echo "SSH to some server you have access to" | $TEE $LOG echo "or" | $TEE $LOG echo "https://_IPA_SERVER_1_/ipa/ui OR https://_IPA_SERVER_2_/ipa/ui; | $TEE $LOG echo " - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -" | $TEE $LOG $CMD2 ipausers --users=$1 | $TEE $LOG if [ -n "$5" ]; then $CMD2 $5 --users=$1 | $TEE $LOG fi if [ -n "$6" ]; then $CMD2 $6 --users=$1 | $TEE $LOG fi if [ -n "$7" ]; then $CMD2 $7 --users=$1 | $TEE $LOG fi if [ -n "$8" ]; then $CMD2 $8 --users=$1 | $TEE $LOG fi if [ -n "$9" ]; then $CMD2 $9 --users=$1 | $TEE $LOG fi echo "See attachment for login information" | $MAIL -s 'New Account Information' -r $EMAIL -a $LOG $4 /bin/rm -f $LOG -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] stubborn old replicas
On 8/28/15 8:17 AM, Vaclav Adamec wrote: You could try this (RH recommended way). It works for me better than cleanallruv.pl <http://cleanallruv.pl/> as this sometimes leads to ldap freeze) unable to decode: {replica 30} 5548fa20001e 5548fa20001e unable to decode: {replica 26} 5548a9a8001a 5548a9a8001a for all of them, on-by-one: ldapmodify -x -D "cn=directory manager" -w XXX dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config changetype: modify replace: nsds5task nsds5task: CLEANRUV30 + On Fri, Aug 28, 2015 at 4:55 PM, Guillermo Fuentes <guillermo.fuen...@modernizingmedicine.com <mailto:guillermo.fuen...@modernizingmedicine.com>> wrote: Hi Janelle, Using the cleanallruv.pl <http://cleanallruv.pl> tool was the only way I was able to get ride of the "unable to decode: {replica x}" entries. This is how I used it, cleaning a replica ID at a time: # For replica id: 40 cleanallruv.pl <http://cleanallruv.pl> -v -D "cn=directory manager" -w - -b 'dc=example,dc=com' -r 40 Note that the "-w -" will make the tool prompt you for the directory manager password. Hope this helps, Guillermo On Thu, Aug 27, 2015 at 10:27 AM, Janelle <janellenicol...@gmail.com <mailto:janellenicol...@gmail.com>> wrote: On 8/27/15 1:05 AM, thierry bordaz wrote: On 08/27/2015 09:41 AM, Ludwig Krispenz wrote: On 08/27/2015 09:08 AM, Martin Kosek wrote: On 08/26/2015 05:31 PM, Simo Sorce wrote: On Wed, 2015-08-26 at 06:36 -0700, Janelle wrote: Hello all, My biggest problem is losing replicas and then trying to delete the entries and rebuild them. Here is a perfect example, I simply can't get rid of these (see below). I have tried (of course after the ORIGINAL "ipa-replica-manage del hostname --force --clean": ipa-replica-manage clean-ruv 25 ldapmodify... with: dn: cn=clean 25, cn=cleanallruv, cn=tasks, cn=config objectclass: extensibleObject replica-base-dn: dc=example,dc=com replica-id: 25 cn: clean 25 And yet nothing works. Any suggestions? This is perhaps the most frustrating part about maintaining IPA. ~J unable to decode: {replica 12} 5588dc2e000c 559f3de60004000c unable to decode: {replica 14} 5587aa8d000e 5587aa8d0003000e unable to decode: {replica 16} 5588f58f0010 55bb7b0800050010 unable to decode: {replica 25} 55a4887b0019 55a4924200040019 unable to decode: {replica 29} 55d199a50001001d 55d199a50001001d unable to decode: {replica 3} 5587c5c30003 55b8a04900010003 unable to decode: {replica 5} 55cc82ab041d0005 55cc82ab041d0005 Have you tried restarting DS before trying to clean the ruv ? I run in a similar problem in a test install recently, and I got better results that way. The bug is known to the DS people and they are working to get out patches that fix the root issue. Simo. CCing DS folks. Wasn't there a recent DS fix that was supposed to improve the RUV situation? Looking at 389 DS Trac, I see some interesting RUV fixes in 1.3.4.x releases: https://fedorahosted.org/389/query?summary=~RUV=closed=milestone=id=summary=status=owner=type=priority=milestone <https://fedorahosted.org/389/query?summary=%7ERUV=closed=milestone=id=summary=status=owner=type=priority=milestone> I see that 389-ds-base-1.3.4.3 is already in Fedora 22+, does the RUV issue happen there? it should not, and I think Thierry verified the fix. The problem we resolved and which we think is the core of the corrupted RUV was that the cleanallruv task did only purge the RUV, but dit not purge the changelog. If cleanallruv was run and the server had a disorderly shutdown (crash or abort when shutdown was hanging) then at restart the changelog RUV was rebuilt from the data in the changelog and if it contained a csn from cleaned RIDs this was added to the RUV (but the reference to the server was lost and so the url part is missing from this RUV. The fix now does remove all references to the cleaned RID from the changelog and the problem should not reoccur with RIDs cleaned with the fix, of course th echangelog can still can contain references to RIDs cleaned before the fix - and if no changelog trimming is configured this is what will happen. So, even after
[Freeipa-users] CA replicas different views???
Hello, I am very confused. I have a couple of data centers and as expected, I have setup CA replicas in each DC. However, this is what makes me nervous/afraid of my configs. In one data center, which sitting on a master and issuing: (as seen from ipa006.example.com) ipa-csreplica-manage list I see ipa002.example.com: master BUT as seen from ipa010.example.com ipa002.example.com: CA not configured How is this possible??? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] stubborn old replicas
On 8/27/15 1:05 AM, thierry bordaz wrote: On 08/27/2015 09:41 AM, Ludwig Krispenz wrote: On 08/27/2015 09:08 AM, Martin Kosek wrote: On 08/26/2015 05:31 PM, Simo Sorce wrote: On Wed, 2015-08-26 at 06:36 -0700, Janelle wrote: Hello all, My biggest problem is losing replicas and then trying to delete the entries and rebuild them. Here is a perfect example, I simply can't get rid of these (see below). I have tried (of course after the ORIGINAL ipa-replica-manage del hostname --force --clean: ipa-replica-manage clean-ruv 25 ldapmodify... with: dn: cn=clean 25, cn=cleanallruv, cn=tasks, cn=config objectclass: extensibleObject replica-base-dn: dc=example,dc=com replica-id: 25 cn: clean 25 And yet nothing works. Any suggestions? This is perhaps the most frustrating part about maintaining IPA. ~J unable to decode: {replica 12} 5588dc2e000c 559f3de60004000c unable to decode: {replica 14} 5587aa8d000e 5587aa8d0003000e unable to decode: {replica 16} 5588f58f0010 55bb7b0800050010 unable to decode: {replica 25} 55a4887b0019 55a4924200040019 unable to decode: {replica 29} 55d199a50001001d 55d199a50001001d unable to decode: {replica 3} 5587c5c30003 55b8a04900010003 unable to decode: {replica 5} 55cc82ab041d0005 55cc82ab041d0005 Have you tried restarting DS before trying to clean the ruv ? I run in a similar problem in a test install recently, and I got better results that way. The bug is known to the DS people and they are working to get out patches that fix the root issue. Simo. CCing DS folks. Wasn't there a recent DS fix that was supposed to improve the RUV situation? Looking at 389 DS Trac, I see some interesting RUV fixes in 1.3.4.x releases: https://fedorahosted.org/389/query?summary=~RUVstatus=closedorder=milestonecol=idcol=summarycol=statuscol=ownercol=typecol=prioritycol=milestone I see that 389-ds-base-1.3.4.3 is already in Fedora 22+, does the RUV issue happen there? it should not, and I think Thierry verified the fix. The problem we resolved and which we think is the core of the corrupted RUV was that the cleanallruv task did only purge the RUV, but dit not purge the changelog. If cleanallruv was run and the server had a disorderly shutdown (crash or abort when shutdown was hanging) then at restart the changelog RUV was rebuilt from the data in the changelog and if it contained a csn from cleaned RIDs this was added to the RUV (but the reference to the server was lost and so the url part is missing from this RUV. The fix now does remove all references to the cleaned RID from the changelog and the problem should not reoccur with RIDs cleaned with the fix, of course th echangelog can still can contain references to RIDs cleaned before the fix - and if no changelog trimming is configured this is what will happen. So, even after the fix old RUVs could pop up and have to be (finally) cleaned. The other source is that these corrupted rivs can be imported from another server by exchanging ruvs in the repl protocol. Cleanallruv tries to address this and to propagate the cleanallruv tasks to all servers it thinks are connected. If there are replication agreements to servers which no longer exist or to servers which cannot be connetcted this will delay the ruv cleaning Hello, I verified the fix in 1.3.4.2 F22 / 389-ds-base-1.3.4.0-6.el7 RHEL7, so after those versions CLEANALLRUV do not create any longer corrupted ruv elements. According to the timestamp in the ruv (for example csn2date.py 5587aa8d0003000e -- 22/06/2015:06:26:21) this are old ruv elements. I think Ludwig is right, these corrupted ruv-elements come from old cleanallruv before the fix was applied. The problem is that even a fixed server can get those corrupted ruv-elements from others servers. All servers in the topology should be updated with that fix, so that at least they stop creating corrupted ruv-elements. Now to get rid of the existing ones, I imagine only brute option of recreating replica and reinit... I hope an other option is possible. thanks thierry Simple question -- what if one is running RHEL 7.1?? Can this fix be installed?? I see you mentioned it is in 1.3.4.0 for RHEL 7, but I don't see it? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] stubborn old replicas
Hello all, My biggest problem is losing replicas and then trying to delete the entries and rebuild them. Here is a perfect example, I simply can't get rid of these (see below). I have tried (of course after the ORIGINAL ipa-replica-manage del hostname --force --clean: ipa-replica-manage clean-ruv 25 ldapmodify... with: dn: cn=clean 25, cn=cleanallruv, cn=tasks, cn=config objectclass: extensibleObject replica-base-dn: dc=example,dc=com replica-id: 25 cn: clean 25 And yet nothing works. Any suggestions? This is perhaps the most frustrating part about maintaining IPA. ~J unable to decode: {replica 12} 5588dc2e000c 559f3de60004000c unable to decode: {replica 14} 5587aa8d000e 5587aa8d0003000e unable to decode: {replica 16} 5588f58f0010 55bb7b0800050010 unable to decode: {replica 25} 55a4887b0019 55a4924200040019 unable to decode: {replica 29} 55d199a50001001d 55d199a50001001d unable to decode: {replica 3} 5587c5c30003 55b8a04900010003 unable to decode: {replica 5} 55cc82ab041d0005 55cc82ab041d0005 -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] freeipa on http?
Going to give this a try today. Thanks so much for taking the time to work this out. ~J On 8/24/15 2:01 AM, Jan Pazdziora wrote: On Thu, Aug 20, 2015 at 02:26:43PM +0200, Jan Pazdziora wrote: On Tue, Aug 18, 2015 at 02:58:50PM -0700, Janelle wrote: Tried that -- but it gives a blank screen. I will try playing with it some more. At least I know we are thinking in the same ballpark I was able to set this up just fine with freeipa-server-4.1.4-4.fc22.x86_64. You need to disable the # Redirect to the secure port if not displaying an error or retrieving # configuration. RewriteCond %{SERVER_PORT} !^443$ RewriteCond %{REQUEST_URI} !^/ipa/(errors|config|crl) RewriteCond %{REQUEST_URI} !^/ipa/[^\?]+(\.js|\.css|\.png|\.gif|\.ico|\.woff|\.svg|\.ttf|\.eot)$ RewriteRule ^/ipa/(.*) https://ipa.example.test/ipa/$1 [L,R=301,NC] part on the IPA server or you will get infinite redirection loop. Also you will need to test it through that SSL proxy, not directly against http://ipa.example.test/, or authentication on the WebUI will not work -- the session cookie is marked as Secure so the browser will not store it when it comes via http, plus the UI checks referer to start with https://. I've put the notes about the setup I've tried to http://www.adelton.com/freeipa/freeipa-behind-ssl-proxy -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] FreeIPA state - performace, commercial usage
I would have to throw in a comment. As someone who has a 16 server cluster with 10,000+ clients and growing, the hardest part is having to tune dirsrv on each and every server. Beyond that, the rest is pretty solid. Perhaps in the 5.x series they would consider adding a way to tune the primary dirsrv at installation time, and have it copy that config via ipa-replica-install or similar. ~Janelle On 8/21/15 4:44 AM, Loris Santamaria wrote: Hi, FWIW one of our customers (a bank) uses freeIPA 3.0 + samba with 4 servers and 5000+ clients, with no major issues. We were able to solve every issue they had tuning the dirsrv or with help from this list. Best regards El vie, 21-08-2015 a las 04:44 +0200, Vaclav Adamec escribió: Hi, Don't want to start flame, but my question is quite simple, is there anybody who use it in real production/commercial setup without any major issues ? don't you lack commercial support ? no issues with auditors ? after a year/two of usage/testing/troubleshooting of freeipa/redhat ipa it seems, for me as a simple admin, to be still not very mature project, even basic configuration isn't very stable/solid to use it in real production. I started with latest freeipa on fedora with one server (VM vmware), then add other master replicas but after many issues I carefully keep one server on redhat 7 with up2date version of ipa from rhel repos, default installation setup, no replication. But still with stability issue (processes died occasionally, mostly due multiple clients removing, sometimes it dies completely with cryptic errors in journal (but sometimes no errors at all just wait for something during restart) and only fast option is restore from snaphot backups with loosing some clients). Performance is also issue, we cannot register more then 4-5 servers at once, or it will timeout (but no visible network or cpu/mem load issue). As there are no other complex solutions like IPA it's quite hard decide what to use as a replacement, but right now it's seems that we have no other option and we probably switch to simple openldap and missing functionality cover by puppet and some 2factor solution. We don't need anything special, no dns handling, no certificates, no AD connection, just simple servers/clients, users with groups and rules for access/sudo. Multimaster (with DNS SRV) solution for higher performance and reliability would be nice, but not necessary if we can keep it stable and handle more clients registration. We have tens of users/groups, hundreds servers/clients with random registration burst as we use it also for temp. build environments and OpenStack instances. Oficial support from RedHat is not very helpful, also they don't provide any real training for IPA, so only option is mail conference (very helpful, thanks for that) and tones of documentation/examples for variety of versions, but for such complex thing probably not enough for commercial use. Can I ask you for your opinion ? Vasek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Cannot uninstall ipa-server
ipa-server-install --uninstall --unattended ~J On 8/19/15 7:41 AM, bahan w wrote: Hello. After an unsuccessfull installation of ipa-server, 3.0.0-42, I try to uninstall it, but the uninstallation hangs at the following step : ### ipa-server-install --uninstall This is a NON REVERSIBLE operation and will delete all data and configuration! Are you sure you want to continue with the uninstall procedure? [no]: yes Shutting down all IPA services ### It hangs forever. Anyway to perform the uninstallation manually ? I throught I saw a method somewhere concerning the removal of the files contained in the following folders : ### /var/lib/ipa/sysrestore /var/lib/ipa-client/sysrestore ### Is it true ? Best regards. Bahan -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] freeipa on http?
Hi, Is there a way to force freeipa web server to accept http requests and not redirect to https? Reason is simple - offloading SSL to a load balancer on the front end. (this is for web only, not the LDAP or Kerberos) Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] freeipa on http?
Simo, I read your blog sometime ago and do like it. However in this case, this is only for HTTPS, not kerberos, so the names do not have to match. It is for users managing accounts across any number of hosts. But thank you. ~J On 8/18/15 3:02 PM, Simo Sorce wrote: On Tue, 2015-08-18 at 18:01 -0400, Simo Sorce wrote: The load balancer would have to have the exact same name (for the clients) as the IPA server, which may be challenging depending on the network configuration you have. More on that issue here: http://ssimo.org/blog/id_019.html On Tue, 2015-08-18 at 14:58 -0700, Janelle wrote: Tried that -- but it gives a blank screen. I will try playing with it some more. At least I know we are thinking in the same ballpark Thank you ~J On 8/18/15 1:55 PM, Rob Crittenden wrote: Janelle wrote: Hi, Is there a way to force freeipa web server to accept http requests and not redirect to https? Reason is simple - offloading SSL to a load balancer on the front end. (this is for web only, not the LDAP or Kerberos) Thank you ~J You could try disabling the rewrite rules to do this in /etc/httpd/conf.d/ipa-rewrite.conf. rob -- Simo Sorce * Red Hat, Inc * New York -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] freeipa on http?
Tried that -- but it gives a blank screen. I will try playing with it some more. At least I know we are thinking in the same ballpark Thank you ~J On 8/18/15 1:55 PM, Rob Crittenden wrote: Janelle wrote: Hi, Is there a way to force freeipa web server to accept http requests and not redirect to https? Reason is simple - offloading SSL to a load balancer on the front end. (this is for web only, not the LDAP or Kerberos) Thank you ~J You could try disabling the rewrite rules to do this in /etc/httpd/conf.d/ipa-rewrite.conf. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] first time web UI access?
Hi, Apparently no one has ever seen this? :-( ~J On 8/14/15 6:37 AM, Janelle wrote: I am curious if anyone else ever sees a problem with first time IPA WEB UI access and the full screen not loading. It requires a reload sometimes once or twice to get it to load properly. Has anyone seen this before? thank you Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] first time web UI access?
I am curious if anyone else ever sees a problem with first time IPA WEB UI access and the full screen not loading. It requires a reload sometimes once or twice to get it to load properly. Has anyone seen this before? thank you Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] users- ssh keys self service
Hi, So I still have been unable to find the problem with blank screens for users when they login to the gui and can not manage anything other than OTP. Out of the box, vanilla install of FreeOTP on RHEL 7.x and using IPA 4.1.4, a user logs in, you see ALL the fields for a split second, before they go blank and there is no way to bring them back. This is over course frustrating since users can not add their SSH keys. They can change there PW, since that is on the ACTION button, which remains visible. Are there any troubleshooting suggestions for this? I have not customized anything. Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] users- ssh keys self service
AHA!!! The problem is found, but the solution eludes me. Any user migrated in compat mode has the problem. NEW users do not. Thoughts? Ideas? troubleshooting? What do I need to make visible for users to edit their settings? ~J On 8/13/15 9:58 AM, Janelle wrote: Hi, So I still have been unable to find the problem with blank screens for users when they login to the gui and can not manage anything other than OTP. Out of the box, vanilla install of FreeOTP on RHEL 7.x and using IPA 4.1.4, a user logs in, you see ALL the fields for a split second, before they go blank and there is no way to bring them back. This is over course frustrating since users can not add their SSH keys. They can change there PW, since that is on the ACTION button, which remains visible. Are there any troubleshooting suggestions for this? I have not customized anything. Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] FreeIPA user ID differs
I too have seen this same unique bug. My guess is, you have compatibility mode enabled AND you used the GUI to manipulate the group memberships. I have found this to be buggy. Using CLI based commands did not have the same results. However, once the 2 trees - cn=accounts and cn=compat are no longer in sync, I have found the only way to fix this is with ldapmodify commands, since neither the GUI nor the command line tools believe the users are in the groups in question anymore. ~Janelle On 8/4/15 2:26 AM, Christopher Lamb wrote: Markus Have you checked both the cn=accounts and cn=compat trees?. Users and groups are stored in both, and both would need manipulation... Ciao Chris From: markus@mc.ingenico.com To: freeipa-users@redhat.com Date: 04.08.2015 11:14 Subject:[Freeipa-users] FreeIPA user ID differs Sent by:freeipa-users-boun...@redhat.com Hi @all, I´ve encountered a strange „error“. I´ve created a user with a generated UID from the predefined range. After creation I´ve had to manipulate the UID to fit an old NIS configuration and set the UID to the old NIS value. FreeIPA shows the correct UID as well as ldapsearch. But if I logon onto a host and enter `id username` I receive the old UID, GID and groups information instead of the corrected one. Maybe someone can help me out to pinpoint the error and to fix it. Cheers, Markus-- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] approving certs?
Hello, Well, I am more used to working with openssl directly, so I am a little confused when using FreeIPA and certmonger. I assume that when a certificate is in this state: status: NEWLY_ADDED_NEED_KEYINFO_READ_PIN stuck: yes That it needs to be approved, but I am not sure where that is. I see all the cert commands, but don't see anything relating to approvals? Am I missing something obvious here? Thank you ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Adding SAN to default self-signed cert?
Trying to figure this out: ipa host-add haproxy.example.com ipa service-add HTTP/haproxy.example@example.com ipa service-add LDAP/haproxy.example@example.com ipa-getcert request -d /tmp -n haproxy-cert -K LDAP/haproxy.example.com -N 'CN=haproxy.example.com,O=EXAMPLE.COM ^ this is where I am confused, because if I created a cert request for the new service, then why am I putting the name of the haproxy in the SAN? Unless I am completely misreading your suggestion? Thank you ~J On 8/2/15 8:53 PM, Fraser Tweedale wrote: On Sun, Aug 02, 2015 at 02:59:52PM -0700, Janelle wrote: Hello everyone, I was wondering if anyone knows of a way to add SAN(s) to the self-signed certificate that are installed when you installed freeipa? Or am I stuck having to do a re-install and use new certificates? If you try to run haproxy as a load balancer in front of the ldap/http servers, well, as you might guess the haproxy server name needs to be added somehow to the server configs so it is a SAN of the existing self-signed certs. I can't think of any way to do it, but maybe some of the pki experts here have any idea? Thank you ~Janelle You do not need a SAN on the root certificate, but on the service certificates. This is supported: you first need to create a service principal for the load balancer, then issue a new service certificate with the haproxy SAN in the CSR (the getcert `-D' option can be used to add a SAN to a certmonger request). HTH, Fraser -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Keeping a Tuesday fun - replication? without replication?
Hello again, Just to keep your Tuesday fun, is this possible: 16 servers. ipa-replica-manage list shows all 16 1 of the servers broke a couple of weeks ago and was removed with clean-ruv but STILL shows up in the replica list, but not a single master has a replica agreement with it, so there is no way to delete it since trying to do ipa-replica-manage del with any options, including force, from ANY servers says there is no replica agreement. How is this possible and how do I get rid of the phantom replica? and I did try --cleanup and it took it, but did nothing. And there is NOTHING in the logs?? To further clarify, it is not a CA either, and never was. Very confusing indeed. I just like to keep the developers on their toes. :-) ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Keeping a Tuesday fun - replication? without replication?
On 8/4/15 9:06 AM, Ludwig Krispenz wrote: On 08/04/2015 05:40 PM, Rob Crittenden wrote: Janelle wrote: Hello again, Just to keep your Tuesday fun, is this possible: 16 servers. ipa-replica-manage list shows all 16 1 of the servers broke a couple of weeks ago and was removed with clean-ruv but STILL shows up in the replica list, but not a single master has a replica agreement with it, so there is no way to delete it since trying to do ipa-replica-manage del with any options, including force, from ANY servers says there is no replica agreement. How is this possible and how do I get rid of the phantom replica? and I did try --cleanup and it took it, but did nothing. And there is NOTHING in the logs?? To further clarify, it is not a CA either, and never was. Very confusing indeed. I just like to keep the developers on their toes. :-) don't know if I want to know the answer, but is it contained in the ruvs ? No. That is why I am baffled. I want to re-add the server to help with loading, but obviously if it still shows up - so weird. Looks like ldapmodify is going to be required. I don't even have any strange CSN/replicas that can't be decoded in list-ruv ~J list shows the those entries in cn=masters,cn=ipa,cn=etc,$SUFFIX. It doesn't show agreements or topology. What output do you see when --cleanup is used? You should check the 389-ds access log after this is run as well to see what searches and mods were attempted. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Adding SAN to default self-signed cert?
Hello everyone, I was wondering if anyone knows of a way to add SAN(s) to the self-signed certificate that are installed when you installed freeipa? Or am I stuck having to do a re-install and use new certificates? If you try to run haproxy as a load balancer in front of the ldap/http servers, well, as you might guess the haproxy server name needs to be added somehow to the server configs so it is a SAN of the existing self-signed certs. I can't think of any way to do it, but maybe some of the pki experts here have any idea? Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Admin password not accepted during replica install
What is in the logs on the machine that is failing? Can you login to admin from anywhere? Logs are you best friend. Also, a simply ssh -vvv will help. ~J On 8/1/15 12:51 PM, Matt . wrote: Hi, This didn't fix it yet. I wonder if there are any checks I can do as in the very past I was able to do a simple replica without any issues. Matt 2015-08-01 21:34 GMT+02:00 Janelle janellenicol...@gmail.com: Double check you do not have AllowGroups set in your /etc/ssh/sshd_config file. If you do, add the admins group. Also, make sure on the master, that the /etc/nsswitch.conf was properly updated. Several server installs I have done, have left off the sss for passwd, group and shadow. passwd: files sss shadow: files sss group: files sss I bet one of those will fix your problem. Restart sssd and/of sshd if you have to make changes. ~Janelle On 8/1/15 10:13 AM, Matt . wrote: Hi Guys, I'm doing a replica install there my admin password for the SSH check to the master is not accepted. The password is not expired, I can use it on the GUI and even changing it in the GUI doesn't fix this. What can I check ? Cheers, Matt -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Admin password not accepted during replica install
lastly -- on the master - do you get the same error if you kinit admin? ~J On 8/1/15 1:05 PM, Matt . wrote: This actually the most important part, and the GSS Failure concerns me: debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /root/.ssh/id_rsa ((nil)), debug2: key: /root/.ssh/id_dsa ((nil)), debug2: key: /root/.ssh/id_ecdsa ((nil)), debug2: key: /root/.ssh/id_ed25519 ((nil)), debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-keyex debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-keyex debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug2: we did not send a packet, disable method debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug1: Unspecified GSS failure. Minor code may provide more information debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /root/.ssh/id_rsa debug3: no such identity: /root/.ssh/id_rsa: No such file or directory debug1: Trying private key: /root/.ssh/id_dsa debug3: no such identity: /root/.ssh/id_dsa: No such file or directory debug1: Trying private key: /root/.ssh/id_ecdsa debug3: no such identity: /root/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: /root/.ssh/id_ed25519 debug3: no such identity: /root/.ssh/id_ed25519: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password admin@ipa-01.domain.local's password: debug3: packet_send2: adding 64 (len 58 padlen 6 extra_pad 64) debug2: we sent a password packet, wait for reply debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password Permission denied, please try again. 2015-08-01 22:02 GMT+02:00 Janelle janellenicol...@gmail.com: What is in the logs on the machine that is failing? Can you login to admin from anywhere? Logs are you best friend. Also, a simply ssh -vvv will help. ~J On 8/1/15 12:51 PM, Matt . wrote: Hi, This didn't fix it yet. I wonder if there are any checks I can do as in the very past I was able to do a simple replica without any issues. Matt 2015-08-01 21:34 GMT+02:00 Janelle janellenicol...@gmail.com: Double check you do not have AllowGroups set in your /etc/ssh/sshd_config file. If you do, add the admins group. Also, make sure on the master, that the /etc/nsswitch.conf was properly updated. Several server installs I have done, have left off the sss for passwd, group and shadow. passwd: files sss shadow: files sss group: files sss I bet one of those will fix your problem. Restart sssd and/of sshd if you have to make changes. ~Janelle On 8/1/15 10:13 AM, Matt . wrote: Hi Guys, I'm doing a replica install there my admin password for the SSH check to the master is not accepted. The password is not expired, I can use it on the GUI and even changing it in the GUI doesn't fix this. What can I check ? Cheers, Matt -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Admin password not accepted during replica install
which points to the configuration of sssd.conf and/or nsswitch.conf It is in there. If you say there are no AllowGroups in sshd, it has to be in one of those 2 places. ~J On 8/1/15 1:26 PM, Matt . wrote: kinit admin works perfectly, that is such strange. 2015-08-01 22:15 GMT+02:00 Janelle janellenicol...@gmail.com: lastly -- on the master - do you get the same error if you kinit admin? ~J On 8/1/15 1:05 PM, Matt . wrote: This actually the most important part, and the GSS Failure concerns me: debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /root/.ssh/id_rsa ((nil)), debug2: key: /root/.ssh/id_dsa ((nil)), debug2: key: /root/.ssh/id_ecdsa ((nil)), debug2: key: /root/.ssh/id_ed25519 ((nil)), debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-keyex debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-keyex debug1: Next authentication method: gssapi-keyex debug1: No valid Key exchange context debug2: we did not send a packet, disable method debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug1: Unspecified GSS failure. Minor code may provide more information debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /root/.ssh/id_rsa debug3: no such identity: /root/.ssh/id_rsa: No such file or directory debug1: Trying private key: /root/.ssh/id_dsa debug3: no such identity: /root/.ssh/id_dsa: No such file or directory debug1: Trying private key: /root/.ssh/id_ecdsa debug3: no such identity: /root/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: /root/.ssh/id_ed25519 debug3: no such identity: /root/.ssh/id_ed25519: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password admin@ipa-01.domain.local's password: debug3: packet_send2: adding 64 (len 58 padlen 6 extra_pad 64) debug2: we sent a password packet, wait for reply debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password Permission denied, please try again. 2015-08-01 22:02 GMT+02:00 Janelle janellenicol...@gmail.com: What is in the logs on the machine that is failing? Can you login to admin from anywhere? Logs are you best friend. Also, a simply ssh -vvv will help. ~J On 8/1/15 12:51 PM, Matt . wrote: Hi, This didn't fix it yet. I wonder if there are any checks I can do as in the very past I was able to do a simple replica without any issues. Matt 2015-08-01 21:34 GMT+02:00 Janelle janellenicol...@gmail.com: Double check you do not have AllowGroups set in your /etc/ssh/sshd_config file. If you do, add the admins group. Also, make sure on the master, that the /etc/nsswitch.conf was properly updated. Several server installs I have done, have left off the sss for passwd, group and shadow. passwd: files sss shadow: files sss group: files sss I bet one of those will fix your problem. Restart sssd and/of sshd if you have to make changes. ~Janelle On 8/1/15 10:13 AM, Matt . wrote: Hi Guys, I'm doing a replica install there my admin password for the SSH check to the master is not accepted. The password is not expired, I can use it on the GUI and even changing it in the GUI doesn't fix this. What can I check ? Cheers, Matt -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] OTP and Laptops
Depending on the laptop -- assuming you are trying to kinit from a terminal window, check the version of Kerberos. It needs to be at least 1.6. ~J On 7/27/15 7:48 AM, John Johnson wrote: Hello, I'm wondering where/how I could get some more information about the underpinnings of the OTP token mechanisms? Ultimately, I'd like to understand the reason why OTP in FreeIPA doesn't work at the moment with laptops, specifically. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Primary certificates
Good morning, I was wondering, I install my servers with the self-signed certs. Now my management wants me to use official certificates. Is there an easy/recommended way to swap out all the certificates on all the servers? Especially with 16 servers, just trying to figure out if this is something I could script with PSSH or similar in order to do them all at once. Does it matter the order? Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] KRA? 4.2?
Hello, I see 4.2 is released today with lots of cool new features. I think I understand the new Vault, but am not familiar with KRA? Wondering if there might be some information on what this is? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] strange password error..
Hello all, Is there any known bug that would cause: Password change failed. Server message: Current password's minimum life has not expired Here is the environment/process (7.1 with IPA 4.1.4) -- 1. reset a user's PW so they are forced to change it. 2. they login and get the Your password has expired... message 3. They are then asked to change it and enter a new PW (twice) 4. This error message pops up, BUT -- the password is still changed. ??? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] blank user screen? (web UI)
On 6/27/15 6:34 AM, Dmitri Pal wrote: On 06/22/2015 01:11 PM, Petr Vobornik wrote: On 06/22/2015 06:39 PM, Janelle wrote: On 6/22/15 9:25 AM, Petr Vobornik wrote: On 06/22/2015 04:15 PM, Janelle wrote: On 6/22/15 5:15 AM, Petr Vobornik wrote: On 06/21/2015 08:35 AM, Janelle wrote: Hi, Sure. Just login as a normal user to the WEB UI. screen is blank: Of course, if you click on Actions - you will see those and you can click on them, but you can't do anything else. This is a vanilla server install, nothing fancy. Oh and there is no error message at all. Any browser = same results. Tried clearing cache, history, web data.. Everything. Many of my users report the same thing. This is 7.1 with IPA 4.1.7 Now the funny part - login as admin and everything works fine. But I certainly can't have everyone logging in as admin. :-) ~Janelle Do you see any error in browser console? Does this happen also to a user which doesn't have any RBAC role assigned(either directly or indrectly)? AHA -- perhaps a clue: [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~J These errors are expected. First two happens when user is not yet authenticated. Third line is just about file for jquery debugging which is not shipped with ipa. Could you inspect other json request? Mainly the 3 which are executed on navigating to user details page (or after clicking on refresh button on the page). Does the first result of first request (of the three) contain user data as in https://pvoborni.fedorapeople.org/images/user_response_data.png I'm unable to reproduce the issue with ipa-server-4.1.0-18.el7_1.3.x86_64. Do these users have some special permissions/roles/rights? The user I did the same from is a User Administrator, however, all the other users are NOT. And if you watch closely, all the details do flash the screen, but then disappear. Refresh does nothing. The one thing - it works flawlessly for admin account. versions (I believe in the newest -- perhaps a bad idea) freeipa-client-4.1.4-1.el7.centos.x86_64 freeipa-server-4.1.4-1.el7.centos.x86_64 freeipa-python-4.1.4-1.el7.centos.x86_64 on a user screen after login - : [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~Janelle If I understand it correctly, you get bunch of 401 Unauthorized errors after successful auth? This should not happen. I have seen something similar when clients were couple minutes in a future than the ipa server (assuming forms based auth is used, otherwise it would fail on obtaining TGT) because session expires immediately if clients are more than 20 mins ahead. Or when krb ticket TTL was less than 5 minutes. Are there any 200 Success requests to ipa/session/json or ipa/session/login_password in the network tab as shown on image: https://pvoborni.fedorapeople.org/images/user_response_data.png after successful login? Was this resolved or we need to file a ticket to track some bug? Just checking back in if anyone was ever able to replicate this or find anything else for me to look for? The Web UI is still useless for my users. :-( ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] blank user screen? (web UI)
On 6/27/15 6:34 AM, Dmitri Pal wrote: On 06/22/2015 01:11 PM, Petr Vobornik wrote: On 06/22/2015 06:39 PM, Janelle wrote: On 6/22/15 9:25 AM, Petr Vobornik wrote: On 06/22/2015 04:15 PM, Janelle wrote: On 6/22/15 5:15 AM, Petr Vobornik wrote: On 06/21/2015 08:35 AM, Janelle wrote: Hi, Sure. Just login as a normal user to the WEB UI. screen is blank: Of course, if you click on Actions - you will see those and you can click on them, but you can't do anything else. This is a vanilla server install, nothing fancy. Oh and there is no error message at all. Any browser = same results. Tried clearing cache, history, web data.. Everything. Many of my users report the same thing. This is 7.1 with IPA 4.1.7 Now the funny part - login as admin and everything works fine. But I certainly can't have everyone logging in as admin. :-) ~Janelle Do you see any error in browser console? Does this happen also to a user which doesn't have any RBAC role assigned(either directly or indrectly)? AHA -- perhaps a clue: [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~J These errors are expected. First two happens when user is not yet authenticated. Third line is just about file for jquery debugging which is not shipped with ipa. Could you inspect other json request? Mainly the 3 which are executed on navigating to user details page (or after clicking on refresh button on the page). Does the first result of first request (of the three) contain user data as in https://pvoborni.fedorapeople.org/images/user_response_data.png I'm unable to reproduce the issue with ipa-server-4.1.0-18.el7_1.3.x86_64. Do these users have some special permissions/roles/rights? The user I did the same from is a User Administrator, however, all the other users are NOT. And if you watch closely, all the details do flash the screen, but then disappear. Refresh does nothing. The one thing - it works flawlessly for admin account. versions (I believe in the newest -- perhaps a bad idea) freeipa-client-4.1.4-1.el7.centos.x86_64 freeipa-server-4.1.4-1.el7.centos.x86_64 freeipa-python-4.1.4-1.el7.centos.x86_64 on a user screen after login - : [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~Janelle If I understand it correctly, you get bunch of 401 Unauthorized errors after successful auth? This should not happen. I have seen something similar when clients were couple minutes in a future than the ipa server (assuming forms based auth is used, otherwise it would fail on obtaining TGT) because session expires immediately if clients are more than 20 mins ahead. Or when krb ticket TTL was less than 5 minutes. Are there any 200 Success requests to ipa/session/json or ipa/session/login_password in the network tab as shown on image: https://pvoborni.fedorapeople.org/images/user_response_data.png after successful login? Was this resolved or we need to file a ticket to track some bug? Still not resolved. Sorry I got sidetracked on other issues this week - namely Marriage Equality -- Yay! :-) ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Crazy Cert problem?
On 6/22/15 7:37 AM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 2:00 PM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:21 AM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:14 AM, Rob Crittenden wrote: Janelle wrote: Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? What version of IPA and distro? (I don't think that probably has anything to do with it, just curious in case it does eventually matter). What does /etc/openldap/ldap.conf look like? Normally it should have TLS_CACERT /etc/ipa/ca.crt Any chance you can share the server and client install logs? rob 4.1.4 = IPA CentOS 7.1 Oooh... Found something: /etc/openldap/ldap.conf: TLS_CACERTDIR/etc/openldap/certs Going to investigate. ~J That should be fine assuming there aren't any certs in there (and on a brand new system I'd think you'd have empty NSS databases). rob So this gets interesting now... Say you have 6 IPA servers, named ipa001-ipa006.example.com -- all working fine. Something happens to 002. It dies. You ipa-replica-manage del --clean --force ipa002 to get rid of it. A period of time, say a month, goes by. You have lost a couple of other replicas for whatever reason, say 3 and 6. You decide you want to rebuild. You start with 002 - leaving the others up and running because you have users working. You firewall off 002 why you rebuild it. You reinstall OS, reinstall FreeIPA. But no matter what, when you start to configure IPA it comes up with the error of being untrusted. Now, you try the same thing on 003 and 006. SAME problem. For fun - you shutdown 005 and uninstall freeipa --unattended and then try to re-install it. Guess what - no issues. Is this somehow related to: Same domain and realm names floating around the net - so is it querying for a name somehow and one of the still running servers is saying - NO NO NO -- that CERT is revoked!!! - even though it never tries to connect to that server. Or am I just thinking far too outside the box? And this is exactly what has happened. Rebuilding one of the servers that was never REMOVED is working just fine. You just jumped to a completely different scenario: from a fresh standalone install to a replica install. We should probably pick one and solve it. I think the leap you're making is that the issue is that it notices some previous cert. A revoked service cert wouldn't have any effect as those service certs aren't in use. It very well could be finding the wrong realm based on DNS SRV records. The logs should show you what the client discovered. Things happen in multiple steps so perhaps there is a disconnect where the right server is used in some, but not all, cases. rob ALL the problems were all related. Even after building brand new servers, the problem persisted and then started cropping up with client installs. The solution traced to bad NSS packages. A simple yum downgrade nss nss-sysinit nss-tools solved it.. Something is up with the 3.18 verion and downgrading to 3.16 seems to have resolved. Should have known it would all be related to an upgrade. Sometimes a slightly older version is best. ~Janelle Can you open a bugzilla about this? rob This looks like the fix - besides downgrading: https://bugzilla.mozilla.org/show_bug.cgi?id=1132941 -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] blank user screen? (web UI)
On 6/22/15 9:25 AM, Petr Vobornik wrote: On 06/22/2015 04:15 PM, Janelle wrote: On 6/22/15 5:15 AM, Petr Vobornik wrote: On 06/21/2015 08:35 AM, Janelle wrote: Hi, Sure. Just login as a normal user to the WEB UI. screen is blank: Of course, if you click on Actions - you will see those and you can click on them, but you can't do anything else. This is a vanilla server install, nothing fancy. Oh and there is no error message at all. Any browser = same results. Tried clearing cache, history, web data.. Everything. Many of my users report the same thing. This is 7.1 with IPA 4.1.7 Now the funny part - login as admin and everything works fine. But I certainly can't have everyone logging in as admin. :-) ~Janelle Do you see any error in browser console? Does this happen also to a user which doesn't have any RBAC role assigned(either directly or indrectly)? AHA -- perhaps a clue: [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~J These errors are expected. First two happens when user is not yet authenticated. Third line is just about file for jquery debugging which is not shipped with ipa. Could you inspect other json request? Mainly the 3 which are executed on navigating to user details page (or after clicking on refresh button on the page). Does the first result of first request (of the three) contain user data as in https://pvoborni.fedorapeople.org/images/user_response_data.png I'm unable to reproduce the issue with ipa-server-4.1.0-18.el7_1.3.x86_64. Do these users have some special permissions/roles/rights? The user I did the same from is a User Administrator, however, all the other users are NOT. And if you watch closely, all the details do flash the screen, but then disappear. Refresh does nothing. The one thing - it works flawlessly for admin account. versions (I believe in the newest -- perhaps a bad idea) freeipa-client-4.1.4-1.el7.centos.x86_64 freeipa-server-4.1.4-1.el7.centos.x86_64 freeipa-python-4.1.4-1.el7.centos.x86_64 on a user screen after login - : [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] blank user screen? (web UI)
On 6/22/15 5:15 AM, Petr Vobornik wrote: On 06/21/2015 08:35 AM, Janelle wrote: Hi, Sure. Just login as a normal user to the WEB UI. screen is blank: Of course, if you click on Actions - you will see those and you can click on them, but you can't do anything else. This is a vanilla server install, nothing fancy. Oh and there is no error message at all. Any browser = same results. Tried clearing cache, history, web data.. Everything. Many of my users report the same thing. This is 7.1 with IPA 4.1.7 Now the funny part - login as admin and everything works fine. But I certainly can't have everyone logging in as admin. :-) ~Janelle Do you see any error in browser console? Does this happen also to a user which doesn't have any RBAC role assigned(either directly or indrectly)? AHA -- perhaps a clue: [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (json, line 0) [Error] Failed to load resource: the server responded with a status of 401 (Unauthorized) (login_kerberos, line 0) [Error] Failed to load resource: the server responded with a status of 404 (Not Found) (jquery-2.0.3.min.map, line 0) ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Crazy Cert problem?
On 6/17/15 2:00 PM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:21 AM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:14 AM, Rob Crittenden wrote: Janelle wrote: Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? What version of IPA and distro? (I don't think that probably has anything to do with it, just curious in case it does eventually matter). What does /etc/openldap/ldap.conf look like? Normally it should have TLS_CACERT /etc/ipa/ca.crt Any chance you can share the server and client install logs? rob 4.1.4 = IPA CentOS 7.1 Oooh... Found something: /etc/openldap/ldap.conf: TLS_CACERTDIR/etc/openldap/certs Going to investigate. ~J That should be fine assuming there aren't any certs in there (and on a brand new system I'd think you'd have empty NSS databases). rob So this gets interesting now... Say you have 6 IPA servers, named ipa001-ipa006.example.com -- all working fine. Something happens to 002. It dies. You ipa-replica-manage del --clean --force ipa002 to get rid of it. A period of time, say a month, goes by. You have lost a couple of other replicas for whatever reason, say 3 and 6. You decide you want to rebuild. You start with 002 - leaving the others up and running because you have users working. You firewall off 002 why you rebuild it. You reinstall OS, reinstall FreeIPA. But no matter what, when you start to configure IPA it comes up with the error of being untrusted. Now, you try the same thing on 003 and 006. SAME problem. For fun - you shutdown 005 and uninstall freeipa --unattended and then try to re-install it. Guess what - no issues. Is this somehow related to: Same domain and realm names floating around the net - so is it querying for a name somehow and one of the still running servers is saying - NO NO NO -- that CERT is revoked!!! - even though it never tries to connect to that server. Or am I just thinking far too outside the box? And this is exactly what has happened. Rebuilding one of the servers that was never REMOVED is working just fine. You just jumped to a completely different scenario: from a fresh standalone install to a replica install. We should probably pick one and solve it. I think the leap you're making is that the issue is that it notices some previous cert. A revoked service cert wouldn't have any effect as those service certs aren't in use. It very well could be finding the wrong realm based on DNS SRV records. The logs should show you what the client discovered. Things happen in multiple steps so perhaps there is a disconnect where the right server is used in some, but not all, cases. rob ALL the problems were all related. Even after building brand new servers, the problem persisted and then started cropping up with client installs. The solution traced to bad NSS packages. A simple yum downgrade nss nss-sysinit nss-tools solved it.. Something is up with the 3.18 verion and downgrading to 3.16 seems to have resolved. Should have known it would all be related to an upgrade. Sometimes a slightly older version is best. ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] blank user screen? (web UI)
Hi, Sure. Just login as a normal user to the WEB UI. screen is blank: Of course, if you click on Actions - you will see those and you can click on them, but you can't do anything else. This is a vanilla server install, nothing fancy. Oh and there is no error message at all. Any browser = same results. Tried clearing cache, history, web data.. Everything. Many of my users report the same thing. This is 7.1 with IPA 4.1.7 Now the funny part - login as admin and everything works fine. But I certainly can't have everyone logging in as admin. :-) ~Janelle On 6/20/15 11:21 PM, Prashant Bapat wrote: Can you share the steps to reproduce this and the error message? On 21 June 2015 at 02:33, Janelle janellenicol...@gmail.com mailto:janellenicol...@gmail.com wrote: Just wondering if others have run into the user login to the web-UI and with the exception of the top part of the screen and menu, all the user details go blank. This makes it hard for a user to click on add ssh key since they can't see it. Have reproduced this dozens of times on all browsers. Very confusing. There must be an answer or known fix? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] blank user screen? (web UI)
Just wondering if others have run into the user login to the web-UI and with the exception of the top part of the screen and menu, all the user details go blank. This makes it hard for a user to click on add ssh key since they can't see it. Have reproduced this dozens of times on all browsers. Very confusing. There must be an answer or known fix? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Installing replica w/o CA?
Maybe this is an obvious question - but I am missign the simple answer. If you create a master and want to create 3 replicas -- creating the first replica works just fine, but I want the 2nd replica chained off the first, and NOT the master. But unless you install a CA on that first replica, you get an error. 1. install master 2. ipa-replica-prepare -- rep001 -- copy file to rep001 3. ipa-replica-install on rep001 4. ipa-replica-prepare rep002 --- does not work saying you can only create replica from master? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] ipa replica failure
On 6/19/15 11:22 AM, Andrew E. Bruno wrote: Hello, First time trouble shooting an ipa server failure and looking for some guidance on how best to proceed. First some background on our setup: Servers are running freeipa v4.1.0 on CentOS 7.1.1503: - ipa-server-4.1.0-18.el7.centos.3.x86_64 - 389-ds-base-1.3.3.1-16.el7_1.x86_64 3 ipa-servers, 1 first master (rep1) and 2 (rep2, rep3) replicates. The replicates were setup to be ca's (i.e. ipa-replica-install --setup-ca...) We have ~3000 user accounts (~1000 active the rest disabled). We have ~700 hosts enrolled (all installed using ipa-client-install and running sssd). Hosts clients are a mix of centos 7 and centos 6.5. We recently discovered one of our replica servers (rep2) was not responding. A quick check of the dirsrv logs /var/log/dirsrv/slapd-/errors (sanitized): PR_Accept() failed, Netscape Portable Runtime error (Process open FD table is full.) ... The server was rebooted and after coming back up had these errors in the logs: 389-Directory/1.3.3.1 B2015.118.1941 replica2:636 (/etc/dirsrv/slapd-) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - Serious Error---Failed to trickle, err=-30973 (BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - Serious Error---Failed in deadlock detect (aborted at 0x0), err=-30973 (BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - Serious Error---Failed in deadlock detect (aborted at 0x0), err=-30973 (BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - Serious Error---Failed to checkpoint database, err=-30973 (BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - Serious Error---Failed to checkpoint database, err=-30973 (BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery) [16/Jun/2015:10:12:33 -0400] - libdb: BDB0060 PANIC: fatal region error detected; run recovery [16/Jun/2015:10:12:33 -0400] - checkpoint_threadmain: log archive failed - BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery (-30973) [16/Jun/2015:16:24:04 -0400] - 389-Directory/1.3.3.1 B2015.118.1941 starting up [16/Jun/2015:16:24:04 -0400] - Detected Disorderly Shutdown last time Directory Server was running, recovering database. ... [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - replica_check_for_data_reload: Warning: disordely shutdown for replica dc=XXX. Check if DB RUV needs to be updated [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - Force update of database RUV (from CL RUV) - 5577006800030003 [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - Force update of database RUV (from CL RUV) - 556f463200140004 [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - Force update of database RUV (from CL RUV) - 556f4631004d0005 [16/Jun/2015:16:24:15 -0400] slapi_ldap_bind - Error: could not send startTLS request: error -1 (Can't contact LDAP server) errno 111 (Connection refused) [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - agmt=cn=cloneAgreement1-rep2 (rep1:389): Replication bind with SIMPLE auth failed: LDAP error -1 (Can't contact LDAP server) () [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - replica_check_for_data_reload: Warning: disordely shutdown for replica o=ipaca. Check if DB RUV needs to be updated [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - Force update of database RUV (from CL RUV) - 556f46290005005b [16/Jun/2015:16:24:15 -0400] set_krb5_creds - Could not get initial credentials for principal [ldap/rep2] in keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for requested realm) [16/Jun/2015:16:24:15 -0400] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -1 (Can't contact LDAP server) ((null)) errno 111 (Connection refused) [16/Jun/2015:16:24:15 -0400] slapi_ldap_bind - Error: could not perform interactive bind for id [] authentication mechanism [GSSAPI]: error -1 (Can't contact LDAP server) [16/Jun/2015:16:24:15 -0400] NSMMReplicationPlugin - agmt=cn=meTorep1 (rep1:389): Replication bind with GSSAPI auth failed: LDAP error -1 (Can't contact LDAP server) () [16/Jun/2015:16:24:15 -0400] - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=xxx--no CoS Templates found, which should be added before the CoS Definition. [16/Jun/2015:16:24:15 -0400] DSRetroclPlugin - delete_changerecord: could not
Re: [Freeipa-users] Crazy Cert problem?
On 6/17/15 6:14 AM, Rob Crittenden wrote: Janelle wrote: Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? What version of IPA and distro? (I don't think that probably has anything to do with it, just curious in case it does eventually matter). What does /etc/openldap/ldap.conf look like? Normally it should have TLS_CACERT /etc/ipa/ca.crt Any chance you can share the server and client install logs? rob 4.1.4 = IPA CentOS 7.1 Oooh... Found something: /etc/openldap/ldap.conf: TLS_CACERTDIR/etc/openldap/certs Going to investigate. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Crazy Cert problem?
On 6/17/15 6:21 AM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:14 AM, Rob Crittenden wrote: Janelle wrote: Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? What version of IPA and distro? (I don't think that probably has anything to do with it, just curious in case it does eventually matter). What does /etc/openldap/ldap.conf look like? Normally it should have TLS_CACERT /etc/ipa/ca.crt Any chance you can share the server and client install logs? rob 4.1.4 = IPA CentOS 7.1 Oooh... Found something: /etc/openldap/ldap.conf: TLS_CACERTDIR/etc/openldap/certs Going to investigate. ~J That should be fine assuming there aren't any certs in there (and on a brand new system I'd think you'd have empty NSS databases). rob Well I was able to get another server stood up, but now if I go back to the server I was TRYING to set up and add it as a replica: all good to here -- then Check SSH connection to remote master Execute check on remote master Check connection from master to remote replica 'ipa002.example.com': Directory Service: Unsecure port (389): OK Directory Service: Secure port (636): OK Kerberos KDC: TCP (88): OK Kerberos KDC: UDP (88): OK Kerberos Kpasswd: TCP (464): OK Kerberos Kpasswd: UDP (464): OK HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK Connection from master to replica is OK. Connection check OK Using reverse zone(s) 202.161.17.in-addr.arpa. Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Unexpected error - see /var/log/ipareplica-install.log for details: NetworkError: cannot connect to 'ldaps://ipa001.example.com': TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. ipareplica-install.log below: 2015-06-17T13:37:48Z DEBUG stderr= 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/radiusproxy.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/realmdomains.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/role.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/rpcclient.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/selfservice.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/selinuxusermap.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/service.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudocmd.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudocmdgroup.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/sudorule.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/trust.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/user.py' 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipalib/plugins/virtual.py' 2015-06-17T13:37:48Z DEBUG importing all plugin modules in '/usr/lib/python2.7/site-packages/ipaserver/install/plugins'... 2015-06-17T13:37:48Z DEBUG importing plugin module '/usr/lib/python2.7/site-packages/ipaserver/install/plugins/adtrust.py' 2015-06-17T13:37:48Z DEBUG importing
Re: [Freeipa-users] Crazy Cert problem?
On 6/17/15 6:21 AM, Rob Crittenden wrote: Janelle wrote: On 6/17/15 6:14 AM, Rob Crittenden wrote: Janelle wrote: Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? What version of IPA and distro? (I don't think that probably has anything to do with it, just curious in case it does eventually matter). What does /etc/openldap/ldap.conf look like? Normally it should have TLS_CACERT /etc/ipa/ca.crt Any chance you can share the server and client install logs? rob 4.1.4 = IPA CentOS 7.1 Oooh... Found something: /etc/openldap/ldap.conf: TLS_CACERTDIR/etc/openldap/certs Going to investigate. ~J That should be fine assuming there aren't any certs in there (and on a brand new system I'd think you'd have empty NSS databases). rob So this gets interesting now... Say you have 6 IPA servers, named ipa001-ipa006.example.com -- all working fine. Something happens to 002. It dies. You ipa-replica-manage del --clean --force ipa002 to get rid of it. A period of time, say a month, goes by. You have lost a couple of other replicas for whatever reason, say 3 and 6. You decide you want to rebuild. You start with 002 - leaving the others up and running because you have users working. You firewall off 002 why you rebuild it. You reinstall OS, reinstall FreeIPA. But no matter what, when you start to configure IPA it comes up with the error of being untrusted. Now, you try the same thing on 003 and 006. SAME problem. For fun - you shutdown 005 and uninstall freeipa --unattended and then try to re-install it. Guess what - no issues. Is this somehow related to: Same domain and realm names floating around the net - so is it querying for a name somehow and one of the still running servers is saying - NO NO NO -- that CERT is revoked!!! - even though it never tries to connect to that server. Or am I just thinking far too outside the box? And this is exactly what has happened. Rebuilding one of the servers that was never REMOVED is working just fine. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migration error?
On Jun 16, 2015, at 01:56, thierry bordaz tbor...@redhat.com wrote: On 06/16/2015 09:02 AM, Ludwig Krispenz wrote: On 06/16/2015 05:07 AM, Janelle wrote: On 6/15/15 1:12 PM, Rob Crittenden wrote: Janelle wrote: On 6/15/15 6:36 AM, Rob Crittenden wrote: Usually means there is a replication conflict entry. You may be able to get more details on what failed by looking at the LDAP access log of both LDAP servers, though I guess I'd expect this happened locally on the IPA box. Hi again, I have been trying to follow this procedure for replication conflicts regarding nsds5ReplConflict, where I had the two account duplicates, but no matter what, I still get: modifying rdn of entry nsuniqueid=ffc68a41-86e71c6-71714816-fcf248a0+uid=janelle,cn=users,cn=accounts,dc=example,dc=com ldap_rename: Constraint violation additional info: Another entry with the same attribute value already exists (attribute: uid) When I am trying to run the modrdn (ldapmodify) command? Which simply refuses to work. I have been at it for over a week now with no luck. I think this is the last of my issues causing my replication problems. What caused this is that I do have multiple helpdesk personnel that had been updating user accounts. This process has been resolved, but we can't seem to remove the last few duplicates. Any suggestions? Is there a missing step in conflict resolution perhaps? these entries are already a result of conflict resolution, If you add the same entry simultaneously on two servers (meaning add it on A and add it on B (before B has received the replicated add from A), there exist two entries with the same dn, which is not possible. So conflict resolution does not arbitrarily throw one away, but renames it and leaves it to the admin, which on to keep. So you should have one entry uid=janelle,... and one nsuniqueid=+uid=janelle, The error you get is coming from 'uid uniqueness'. Like ludwig mention, it exists duplicated entries with both of them 'uid=janelle'. 'uid uniqueness' plugin prevents you to do a direct MODRDN on one of them because, it finds duplicated 'uid=janelle'. you can delete the nsuniqeid= entry to get rid of it. +1 thierry There is a request to hide these nsuniqueid+uid entries from regular searches, it will be in a next release of 389 Ludwig ~J -- But everything I try to delete fails. Is there a procedure in 389-DS I can read for this? Maybe I am missing an option in ldapmodify? I am happy to delete, if only it would let me. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migration error?
Good morning, Just a quick note. I hope that all my questions do not make any one the DEV Team think that I do not support FreeIPA wholly and completely. I am a huge fan of this package and have in fact discussed with several of my clients (I'm a consultant of course) who have purchased RH support contracts just because of this. The product is wonderful and has potential of being even better as you continue to add new features. Thank you so much for all the support you have provided. I hope RH understands too that many new customers come from recommendations from us consultant-types :-) Ok, so I just wanted to throw that in this thread -- a big THANK YOU to the IPA Team and all the work accomplished so far. You are the best! ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Crazy Cert problem?
Hi, Had a server - named ipa001.example.com -- it was a replica. It died. It was re-installed. However, prior to the re-install it was saying the wonderful: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. It was rebuilt - new OS and doing a brand new ipa-server-install (NOT a replica or trying to join it back in to the existing ring of servers) and at the end of the ipa-server-install - it gives: Done. Restarting the directory server Restarting the KDC Restarting the certificate server Restarting the web server Unable to set admin password Command ''/usr/bin/ldappasswd' '-h' 'ipa001.example.com' '-ZZ' '-x' '-D' 'cn=Directory Manager' '-y' '/var/lib/ipa/tmp5Fxy2Z' '-T' '/var/lib/ipa/tmpnz0jLs' 'uid=admin,cn=users,cn=accounts,dc=example,dc=com'' returned non-zero exit status 1 Configuration of client side components failed! ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'example.com' '--server' 'ipa001.example.com' '--realm' 'example.com' '--hostname' 'ipa001.example.com'' returned non-zero exit status 1 and checking /var/log/ipaclient-install.log - the exact same TLS error But this is a brand new system, with brand new OS and the install was ipa-server-install to install a clean server. I don't understand how this is happening. There is no peer to be not trusted? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Migration error?
Good morning and happy Monday, I have a strange problem. Wondering if anyone has seen this before in trying to run an ipa migrate-ds? ipa: ERROR: The search criteria was not specific enough. Expected 1 and found 2. The migration worked previously, but now, in order to try and update some missing accounts that were added, now it no longer works and generates this error. I can't find anyway to get verbose information to found out what it is finding 2 of? Any help is appreciated. ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migration error?
On 6/15/15 1:12 PM, Rob Crittenden wrote: Janelle wrote: On 6/15/15 6:36 AM, Rob Crittenden wrote: Usually means there is a replication conflict entry. You may be able to get more details on what failed by looking at the LDAP access log of both LDAP servers, though I guess I'd expect this happened locally on the IPA box. Hi again, I have been trying to follow this procedure for replication conflicts regarding nsds5ReplConflict, where I had the two account duplicates, but no matter what, I still get: modifying rdn of entry nsuniqueid=ffc68a41-86e71c6-71714816-fcf248a0+uid=janelle,cn=users,cn=accounts,dc=example,dc=com ldap_rename: Constraint violation additional info: Another entry with the same attribute value already exists (attribute: uid) When I am trying to run the modrdn (ldapmodify) command? Which simply refuses to work. I have been at it for over a week now with no luck. I think this is the last of my issues causing my replication problems. What caused this is that I do have multiple helpdesk personnel that had been updating user accounts. This process has been resolved, but we can't seem to remove the last few duplicates. Any suggestions? Is there a missing step in conflict resolution perhaps? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Migration error?
On 6/15/15 6:36 AM, Rob Crittenden wrote: Janelle wrote: Good morning and happy Monday, I have a strange problem. Wondering if anyone has seen this before in trying to run an ipa migrate-ds? ipa: ERROR: The search criteria was not specific enough. Expected 1 and found 2. The migration worked previously, but now, in order to try and update some missing accounts that were added, now it no longer works and generates this error. I can't find anyway to get verbose information to found out what it is finding 2 of? Usually means there is a replication conflict entry. You may be able to get more details on what failed by looking at the LDAP access log of both LDAP servers, though I guess I'd expect this happened locally on the IPA box. rob I found the problem, but now when trying to re-init from a good server using ipa-replica-manage re-initialize, I get: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user. But how does THIS happen?? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] 4.x on CentOS 6?
Hi everyone, Does anyone know if it is possible to install the 4.1 ipa-CLIENT (not the server - just the client) on a CentOS 6.6 system? My guess is this is really just based on sssd, or am I missing something? I would like to get OTP on 6.6 system, just not sure if that is possible. Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] newer sssd on centos 5?
Has anyone built a newer version of sssd for RHEL/centos 5.x?? Currently only 1.5.x Just wondering if maybe it is limited due to some library or compatibility issues? Thank you Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Successful Install on VB...
By default, fedora has all the ports blocked via firewalld You need to either enable the ports, or disable the firewall. PORTS='80 443 389 636 88 464' for PORT in $PORTS; do firewall-cmd --permanent --zone=public --add-port=$PORT/tcp; done PORTS='88 464 123' for PORT in $PORTS; do firewall-cmd --permanent --zone=public --add-port=$PORT/udp; done firewall-cmd --reload ~J On 6/5/15 12:50 PM, James Benson wrote: Dear all, I recently install Fedora Server 22 on a virtualbox with the ethernet bridged (can successfully ping it, ssh, etc) and I can do a kinit admin and ipa user-add as the instructions detail in the next steps, however, I cannot access the webui. Has anyone else ran into this issue? I've tried to check the services, however, they don't seem to want to start (no errors, just don't see them in the service status menu) Any help would be great as I would greatly like to use the website over commands if possible. Thank you, James -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] how to delete duplicate?
I have a duplicate user. Same exact name, but different UID's. But there does not seem to be a way to do ipa user-del on anything other than username, which ends up returning: # ipa user-del another_username ipa: ERROR: The search criteria was not specific enough. Expected 1 and found 2. Any ideas on how I can delete this user? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Antwort: Re: Haunted servers?
On May 29, 2015, at 00:41, thierry bordaz tbor...@redhat.com wrote: On 05/29/2015 08:16 AM, Christoph Kaminski wrote: freeipa-users-boun...@redhat.com schrieb am 28.05.2015 13:23:26: Von: Alexander Frolushkin alexander.frolush...@megafon.ru An: 'thierry bordaz' tbor...@redhat.com Kopie: freeipa-users@redhat.com freeipa-users@redhat.com Datum: 28.05.2015 13:24 Betreff: Re: [Freeipa-users] Haunted servers? Gesendet von: freeipa-users-boun...@redhat.com Unfortunately, after a couple of minutes, on two of three servers error comes back in little changed form: # ipa-replica-manage list-ruv unable to decode: {replica 16} Before cleanruv it looked like: # ipa-replica-manage list-ruv unable to decode: {replica 16} 548a81260010 548a81260010 And one server seems to be fixed completely. WBR, Alexander Frolushkin we had the same problem (and some more) and yesterday we have successfully cleaned the gohst rid's our fix: Hi Christoph, THanks for sharing this procedure. This bug is difficult to workaround and that is a good idea to write it down. 1. stop all cleanallruv Tasks, if it works with ipa-replica-manage abort-clean-ruv. It hasnt worked here. We have done it manually on ALL replicas with: a) replica stop b) delete all nsds5ReplicaClean from /etc/dirsrv/slapd-HSO/dse.ldif c) replica start Yes the ability to abort clean ruv hits the same retry issue that cleanallruv. It has been addressed with https://fedorahosted.org/389/ticket/48154 2. prepare on EACH ipa a cleanruv ldif file with ALL ghost rids inside (really ALL from all ipa replicas, we has had some rids only on some replicas...) Example: dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config changetype: modify replace: nsds5task nsds5task:CLEANRUV11 dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config changetype: modify replace: nsds5task nsds5task:CLEANRUV22 dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config changetype: modify replace: nsds5task nsds5task:CLEANRUV37 ... It should work but I would prefer to do it in an other order. We need to clean a specific RID, on all replica, at the same time. We do not need to clean all RIDs at the same time. Having several CLEANRUV in parallel for differents RID should work but I do not know how much it has been tested that way. So I would recommend to clean, in parallel on all replicas, RID 11. Then when it is completed, RID 22. Then RID 37. 3. do a ldapmodify -h 127.0.0.1 -D cn=Directory Manager -W -x -f $your-cleanruv-file.ldif on all replicas AT THE SAME TIME :) we used terminator for it (https://launchpad.net/terminator). You can open multiple shell windows inside one window and send to all at the same time the same commands... same remark I would split your-cleanruv-file.ldif into three files cleanruv-11-file.ldif,... 4. we have done a re-initialize of each IPA from our first master Do you mean a total init ? I do not see a real need for that. If you are ready to reinit all replicas, there is a very simple way to get rid of all these ghost RIDs. Select the good master that is having all the updates do a ldif export without the replication data do a ldif import of exported file do online reinit of the full topology, cascading from the good master down to the consumers Most of the time we try to avoid asking a full reinit of the topology because DB are large. 5. restart of all replicas we are not sure about the point 3 and 4. Maybe they are not necessary, but we have done it. If something fails look at defect LDAP entries in whole ldap, we have had some entries with 'nsunique-$HASH' after the 'normal' name. We have deleted them. do you mean entries with 'nsuniqueid' attribute in the RDN. This could be create during replication conflicts when updates are received in parallele on different replicas. thanks thierry MfG Christoph Kaminski -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project Looks like I'll be giving this a try. So glad someone else is seeing exactly the same issues. Hopefully soon we can find the cause. ~J-- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Haunted servers?
On 5/26/15 7:04 AM, thierry bordaz wrote: On 05/26/2015 08:47 AM, Martin Kosek wrote: On 05/26/2015 12:20 AM, Janelle wrote: On 5/24/15 3:12 AM, Janelle wrote: And just like that, my haunted servers have all returned. I am going to just put a gun to my head and be done with it. :-( Why do things run perfectly and then suddenly ??? Logs show little to nothing, mostly because the servers are so busy, they have already rotated out. unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 23} 5545d61f00020017 555432430017 unable to decode {replica 24} 554d53d30018 554d54a400020018 unable to decode {replica 25} 554d78bf0019 555af30200040019 unable to decode {replica 9} 55402c3900030009 55402c3900030009 Don't know what to do anymore. At my wit's end.. ~J So things are getting more interesting. Still trying to find the leaking server(s). here is what I mean by that. As you see, I continue to find these -- BUT, notice a new symptom -- replica 9 does NOT show any other data - it is blank? Hello Janelle, Thanks for update. So you worry that there might still be the rogue IPA replica that would be injecting the wrong replica data? In any case, I bet Ludwig and Thierry will follow up with your thread, there is just delay caused by the various public holidays and PTOs this week and we need to rest before digging into the fun with RUVs - as you already know yourself :-) unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 24} 554d53d300010018 554d54a400020018 unable to decode {replica 25} 554d78bf00020019 555af30200040019 unable to decode {replica 9} Now, if I delete these from a server using the ldapmodify method - they go away briefly, but then if I restart the server, they come back. Let me try to explain -- given a number of servers, say 8, if I user ldapmodify to delete from 1 of those, they seem to go away from maybe 4 of them -- but if I wait a few minutes, it is almost as though replication is re-adding these bad replicas from the servers that I have NOT deleted them from. On each replica (master/replica) there are one RUV in the database and one RUV in the changelog. When cleanallruv succeeds it clears both of them. All replica should be reachable when you issue cleanallruv, so that it can clean the RUVs on all the replicas in almost single operation. If some replica are not reachable, they keep information of about the cleaned RID and then can later propagate those old RID to the rest of the replica. Ludwig managed to reproduce the issue with a quite complex test case (3 replicas and multiple cleanallruv). We have not yet identified the reason how a cleaned replicaId can get resurrected. In parallel we just reproduced it without a clear test case but in a 2 replica topology. After spending well over 2 days trying to clean things -- I am now here: CLEANALLRUV tasks RID 16 Not all replicas finished cleaning, retrying in 14400 seconds RID 19 None RID 22 None What is going on here? All the same data still exists as shown above in the original thread, but I seem to be stuck. I know I am not the only person having replica issues. Is there anything else I can try? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Haunted servers?
On 5/26/15 7:04 AM, thierry bordaz wrote: On 05/26/2015 08:47 AM, Martin Kosek wrote: On 05/26/2015 12:20 AM, Janelle wrote: On 5/24/15 3:12 AM, Janelle wrote: And just like that, my haunted servers have all returned. I am going to just put a gun to my head and be done with it. :-( Why do things run perfectly and then suddenly ??? Logs show little to nothing, mostly because the servers are so busy, they have already rotated out. unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 23} 5545d61f00020017 555432430017 unable to decode {replica 24} 554d53d30018 554d54a400020018 unable to decode {replica 25} 554d78bf0019 555af30200040019 unable to decode {replica 9} 55402c3900030009 55402c3900030009 Don't know what to do anymore. At my wit's end.. ~J So things are getting more interesting. Still trying to find the leaking server(s). here is what I mean by that. As you see, I continue to find these -- BUT, notice a new symptom -- replica 9 does NOT show any other data - it is blank? Hello Janelle, Thanks for update. So you worry that there might still be the rogue IPA replica that would be injecting the wrong replica data? In any case, I bet Ludwig and Thierry will follow up with your thread, there is just delay caused by the various public holidays and PTOs this week and we need to rest before digging into the fun with RUVs - as you already know yourself :-) unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 24} 554d53d300010018 554d54a400020018 unable to decode {replica 25} 554d78bf00020019 555af30200040019 unable to decode {replica 9} Now, if I delete these from a server using the ldapmodify method - they go away briefly, but then if I restart the server, they come back. Let me try to explain -- given a number of servers, say 8, if I user ldapmodify to delete from 1 of those, they seem to go away from maybe 4 of them -- but if I wait a few minutes, it is almost as though replication is re-adding these bad replicas from the servers that I have NOT deleted them from. On each replica (master/replica) there are one RUV in the database and one RUV in the changelog. When cleanallruv succeeds it clears both of them. All replica should be reachable when you issue cleanallruv, so that it can clean the RUVs on all the replicas in almost single operation. If some replica are not reachable, they keep information of about the cleaned RID and then can later propagate those old RID to the rest of the replica. Ludwig managed to reproduce the issue with a quite complex test case (3 replicas and multiple cleanallruv). We have not yet identified the reason how a cleaned replicaId can get resurrected. In parallel we just reproduced it without a clear test case but in a 2 replica topology. So my question is simple - is there something in the logs I can look for that would indicate the SOURCE of these bogus entries? Is the replica 9 with NO extra data any indication of something I could look for? I guess that if I have the answer to your question we would have understood the bug .. A little more information to go on: I changed my password on a master (actually, the original master) and was able to login to each replica within a few seconds with the new password. This tells me replication is working across all the servers. I also created a new account and it showed up on all the servers, again within 15-20 seconds. This tells me replication is working just fine. I don't understand why the cleanallruv does not process across all the servers the same way. Baffling indeed. Perhaps the most important question -- does these bogus entries actually cause a problem? I mean they don't seem to be. What if I just ignored them? ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Haunted servers?
On 5/24/15 3:12 AM, Janelle wrote: And just like that, my haunted servers have all returned. I am going to just put a gun to my head and be done with it. :-( Why do things run perfectly and then suddenly ??? Logs show little to nothing, mostly because the servers are so busy, they have already rotated out. unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 23} 5545d61f00020017 555432430017 unable to decode {replica 24} 554d53d30018 554d54a400020018 unable to decode {replica 25} 554d78bf0019 555af30200040019 unable to decode {replica 9} 55402c3900030009 55402c3900030009 Don't know what to do anymore. At my wit's end.. ~J So things are getting more interesting. Still trying to find the leaking server(s). here is what I mean by that. As you see, I continue to find these -- BUT, notice a new symptom -- replica 9 does NOT show any other data - it is blank? unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 24} 554d53d300010018 554d54a400020018 unable to decode {replica 25} 554d78bf00020019 555af30200040019 unable to decode {replica 9} Now, if I delete these from a server using the ldapmodify method - they go away briefly, but then if I restart the server, they come back. Let me try to explain -- given a number of servers, say 8, if I user ldapmodify to delete from 1 of those, they seem to go away from maybe 4 of them -- but if I wait a few minutes, it is almost as though replication is re-adding these bad replicas from the servers that I have NOT deleted them from. So my question is simple - is there something in the logs I can look for that would indicate the SOURCE of these bogus entries? Is the replica 9 with NO extra data any indication of something I could look for? I am not willing to give up easily (as you might have already guessed) and I am determined to find the cause of these. I know we need more logs, but with all the traffic, the logs rollover within a few hours, and if the problem is happening at 3am for example, I am not able to track it down because the logs have rolled. Back to my investigations. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] Haunted servers?
And just like that, my haunted servers have all returned. I am going to just put a gun to my head and be done with it. :-( Why do things run perfectly and then suddenly ??? Logs show little to nothing, mostly because the servers are so busy, they have already rotated out. unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 22} 55371e9e0016 553eec6400040016 unable to decode {replica 23} 5545d61f00020017 555432430017 unable to decode {replica 24} 554d53d30018 554d54a400020018 unable to decode {replica 25} 554d78bf0019 555af30200040019 unable to decode {replica 9} 55402c3900030009 55402c3900030009 Don't know what to do anymore. At my wit's end.. ~J -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] passwords
I have a question regarding passwords. It seems IPA does a very nice job of generating random passwords. Is there a way to use that feature without actually setting it on a user? Something akin to pwgen? Thank you ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] RUV clean error?
Hi all, Getting close to being clean again, but have this - which is a new error: CLEANALLRUV tasks RID 9 Waiting to process all the updates from the deleted replica... ??? I had done a ipa-replica-manage del --cleanup on the deleted replica (this is related to that duplicate I had) but can't seem to get it to go away. Thoughts? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] RUV clean error?
Restarts on all the servers seem to have resolved all of the issues. Back to a clean environment again. thanks ~J On 5/23/15 9:46 AM, Janelle wrote: Hi all, Getting close to being clean again, but have this - which is a new error: CLEANALLRUV tasks RID 9 Waiting to process all the updates from the deleted replica... ??? I had done a ipa-replica-manage del --cleanup on the deleted replica (this is related to that duplicate I had) but can't seem to get it to go away. Thoughts? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] replication again :-(
On 5/20/15 7:53 AM, Mark Reynolds wrote: On 05/20/2015 10:17 AM, thierry bordaz wrote: On 05/20/2015 03:46 PM, Janelle wrote: On 5/20/15 6:01 AM, thierry bordaz wrote: On 05/20/2015 02:57 AM, Janelle wrote: On 5/19/15 12:04 AM, thierry bordaz wrote: On 05/19/2015 03:42 AM, Janelle wrote: On 5/18/15 6:23 PM, Janelle wrote: Once again, replication/sync has been lost. I really wish the product was more stable, it is so much potential and yet. Servers running for 6 days no issues. No new accounts or changes (maybe a few users changing passwords) and again, 5 out of 16 servers are no longer in sync. I can test it easily by adding an account and then waiting a few minutes, then run ipa user-show --all username on all the servers, and only a few of them have the account. I have now waited 15 minutes, still no luck. Oh well.. I guess I will go look at alternatives. I had such high hopes for this tool. Thanks so much everyone for all your help in trying to get things stable, but for whatever reason, there is a random loss of sync among the servers and obviously this is not acceptable. regards ~J All the replicas are happy again. I found these again: unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 23} 5553e3a30017 555432430017 unable to decode {replica 24} 554d53d30018 554d54a400020018 What I also found to be interesting is that I have not deleted any masters at all, so this was quite perplexing where the orphaned entries came from. However I did find 3 of the replicas did not show complete RUV lists... While most of the replicas had a list of all 16 servers, a couple of them listed only 4 or 5. (using ipa-replica-manage list-ruv) I don't know about the orphaned entries. Did you get entries below deleted parents ? AFAIK all replicas are master and so have an entry {replica rid} in the RUV. We should expect all servers having the same number of RUVelements (16, 4 or 5). The servers with 4 or 5 may be isolated so that they did not received updates from those with 16 RUVelements. would you copy/paste an example of RUV with 16 and with 4-5 ? Now, the steps to clear this were: Removed the unable to decode with the direct ldapmodify's. This worked across all replicas, which was nice and did not have to be repeated in each one. In other words, entered on a single server, and it was removed on all. Hello, Did you do direct ldapmodify onto the RUV entry (nsuniqueid=---,SUFFIX) , clean RUV ? Thierry, Janelle just manually added a cleanallruv task (that I had recommended the other week). Mark dc1-ipa1 and dc1-ipa2 are missing some RUVelement. If you do an update on dc3-ipa1, is it replicated to dc1-ipa[12] ? Also there are duplicated RID (9, 25) for dc1-ipa2.example.com:389. You may see some messages like 'attrlist_replace' in some error logs. 25 seems to be the new RID. thanks thierry re-initialized --from=good server on the ones with the short list. Waited 5 minutes to let everything settle, then started running tests of adds/deletes which seemed to be just fine. Here are 2 of the DCs - Node dc1-ipa1 - dc4-ipa4.example.com 389 21 dc1-ipa1.example.com 389 10 dc1-ipa4.example.com 389 4 - Node dc1-ipa2 - dc4-ipa4.example.com 389 21 dc1-ipa1.example.com 389 10 dc1-ipa2.example.com 389 25 dc1-ipa3.example.com 389 8 dc1-ipa4.example.com 389 4 - Node dc1-ipa3 - dc3-ipa1.example.com 389 14 dc3-ipa2.example.com 389 13 dc3-ipa3.example.com 389 12 dc3-ipa4.example.com 389 11 dc2-ipa1.example.com 389 7 dc2-ipa2.example.com 389 6 dc2-ipa3.example.com 389 5 dc2-ipa4.example.com 389 3 dc4-ipa1.example.com 389 18 dc4-ipa2.example.com 389 19 dc4-ipa3.example.com 389 20 dc4-ipa4.example.com 389 21 dc1-ipa1.example.com 389 10 dc1-ipa2.example.com 389 25 dc1-ipa2.example.com 389 9 dc1-ipa3.example.com 389 8 dc1-ipa4.example.com 389 4 unable to decode {replica 16} 5535647200030010 5535647200030010 unable to decode {replica 24} 554d53d30018 554d54a400020018 dc5-ipa1.example.com 389 26 dc5-ipa2.example.com 389 15 dc5-ipa3.example.com 389 17 - Node dc1-ipa4 - dc3-ipa1.example.com 389 14 dc3-ipa2.example.com 389 13 dc3-ipa3.example.com 389 12 dc3-ipa4.example.com 389 11 dc2-ipa1.example.com 389 7 dc2-ipa2.example.com 389 6 dc2-ipa3.example.com 389 5 dc2-ipa4.example.com 389 3 dc4-ipa1.example.com 389 18 dc4-ipa2.example.com 389 19 dc4-ipa3.example.com 389 20 dc4-ipa4.example.com 389 21 dc1-ipa1.example.com 389 10 dc1-ipa2.example.com 389 25 dc1-ipa2.example.com 389 9 dc1-ipa3.example.com 389 8 dc1-ipa4
Re: [Freeipa-users] replication again :-(
On 5/21/15 5:20 AM, thierry bordaz wrote: Hello Janelle, Those 3 RIDs were already present in Node dc2-ipa1, correct ? They reappeared on others nodes as well ? May be ds2-ipa1 established a replication session with its peers and send those RIDs. Could you track in all the access logs, when the op csn=5552f71800030017 was applied. Note that the two hexa values of replica 23 changed (5545d61f00020017 5552f71800030017 vs 5553e3a30017 555432430017). Have you recreated a replica 23 ?. Do you have replication logging enabled ? thanks thierry Just to help me -- what is the best way to enable the logging level you need? I thought I did it correctly adding to ldif.dse, but I don't think it took. I am used to OpenLDAP, so perhaps there is a different way to do it with 389-ds. Can you suggest settings of logging you want me to use? ~Janelle -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project