Re: [Freeipa-users] Kerberos Clock Skew too great

[Rakesh Rajasekharan]

I was seeing a lot of entries in the krb5kdc.log like below

"krb5kdc[10403](info): TGS_REQ (4 etypes {18 17 16 23}) 10.1.4.219: ISSUE:
authtime 1485450918, etypes {rep=18 tkt=18 ses=18}, host/my-host@MYDOMAIN"

On one env.. where users rarely log in... even there I see a lot of such
requests.


Finally , I think  I was able to track this down..  there are few local
accounts ( non freeipa ) on my hosts . These are used to run some custom
scripts through cron and run frequently ( every few mins ).
So, I feel  whenever thers a request for "su - " or a sudo to
the local user, that would also end up calling the Kerbros service.. and
since it runs so frequently on all the hosts.. they would be choking the
IPA master / replica with so many requests..

Please correct me If I am wrong in the above assumption.

Going by the above logic.. I have added filter_users section with these
users in the sssd.conf . Hopefully I would see a drop in the number of
requests




On Mon, Jan 23, 2017 at 11:27 PM, Robbie Harwood 
wrote:

> Rakesh Rajasekharan  writes:
>
> > one more question I was curious is.. when does the krb5kdc.log get
> entries
> > . .. I mean is it only when someone makes an attempt to login to a server
> > that the log file  krb5kdc.log on the IPA master gets updated or there
> are
> > other scenarios as well
>
> It's controlled by /etc/kdc.conf ; take a look at the "[logging]" section
> in
> `man 5 kdc.conf` for more information.
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Backend & UI plugin update for 4.4.x

[Steve Huston]

Just did it again with the same result.  Reinstalled the machine, then
did a 'yum install ipa-server python2-ipaserver httpd' which pulled in
version 4.4.0-14.el7_3.4 and a bunch of other packages.  Next was the
ipa-server-install as I used before, only without --mkhomedir this
time.  After entering the passwords for directory administrator and
the admin user, I then logged in to the web interface, immediately
clicked "add" and added a user 'foobar'.  When I clicked "add and
edit" and was brought to the user information page, it looks like this
at the bottom:

https://www.dropbox.com/s/e67j8rdaq9wvkni/Screenshot%202017-01-26%2011.33.03.png?dl=0

I then entered an employee number of '0001' just to give something to
save, and clicked save.  The screen now shows this (I've clicked the
drop-down on the manager field so the choices are visible):

https://www.dropbox.com/s/oxmqwf2rsz15grd/Screenshot%202017-01-26%2011.33.58.png?dl=0

Holding shift and clicking reload, the page now looks like this (the
employee number field is also blank again):

https://www.dropbox.com/s/f8ptycnetvsxjnb/Screenshot%202017-01-26%2011.35.03.png?dl=0

Since we do run a repackaged distribution here (Springdale Linux), I
just unpacked ipa-server-common from our repository with the above
version, and 
http://mirror.centos.org/centos/7/updates/x86_64/Packages/ipa-server-common-4.4.0-14.el7.centos.4.noarch.rpm,
and 'diff' found zero differences between them.  Unlikely, but I
wanted to rule out a packaging error causing the problem.

On Wed, Jan 25, 2017 at 4:12 PM, Steve Huston
 wrote:
> No, that should be all of the major changes; the puppet module that
> installs things only puts the two plugin files in their respective
> places.  The client part of the IPA module makes changes to have the
> machine join the domain and whatnot, but those shouldn't affect the
> webui.
>
> I do modify the schema by adding some attribute types for Puppet,
> namely puppetClass, parentNode, environment, puppetVar, and the object
> class puppetClient.  That's basically right from one of the Puppet
> webpages and also worked in the past - and is one of the things the
> python plugin does, add the appropriate objectclass to host entries if
> puppetVar is added to a host entry.
>
> My steps to install:
> * ipa-server-install --realm= --domain= --mkhomedir
> --hostname= --no-host-dns
> * ldapmodify -ZZ -h localhost -x -D 'cn=Directory Manager' -W
>   < paste puppet schema changes>
>   < paste DN entry for uid=hostadder,cn=sysaccounts,cn=etc... - a
> service account used by puppet for adding hosts to IPA >
> * login to web UI
> * * Change home directory base, default shell, default SELinux user
> * * Add SELinux user map for staff/sysadmin users
> * * Add "user adder" permission/privilege/role for users who will be
> able to create stageusers
>
> That's about as far as I got before I realized some of the plugin
> pieces weren't working, and then fixed the python plugin followed by
> working on the UI plugin and finding this problem.  I'll go wipe and
> reinstall the system again and walk through the steps, but test the UI
> first and in between to see if I can find which of the steps might be
> causing things to hiccup.
>
> On Wed, Jan 25, 2017 at 1:42 PM, Pavel Vomacka  wrote:
>> Hello Steve,
>>
>> I tried to reproduce what you described on the very same version of
>> ipa-server and I was not successful. Actually I was not used your back-end
>> plugin. I tried it with no plugin and then with your UI plugin and both
>> worked correctly. Did you do any other changes somewhere in your
>> installation?
>>
>> I will try it again also with your Python plugin and we'll see.
>>
>>
>> On 01/24/2017 08:59 PM, Steve Huston wrote:
>>>
>>> And now I'm convinced this has nothing to do with my plugin and
>>> instead is a bug somewhere in FreeIPA.
>>>
>>> I removed the entirety of the "astrocustom" plugin that I wrote,
>>> restarted httpd, and force reloaded the page in chrome.  I clicked to
>>> add a new user, gave the basic information, and clicked "add and
>>> edit".  The bottom of the page shows the "Employee information" on the
>>> left side bottom, and the manager drop-down is empty.  I entered '1'
>>> in the "employee type" field and clicked save, and now "Employee
>>> Information" is on the right side directly under "Contact settings",
>>> and the manager drop-down is populated with the list of UIDs on the
>>> system.
>>>
>>> When the UI is in the failed state, the "email address" field is also
>>> blank, but when things switch to how they should be (after submitting
>>> a change) it is populated with the email address in the record.  I
>>> just tested by adding a telephone number to the record, and that also
>>> made the contact information and employee information facets refresh
>>> with the proper data.  Pressing shift-reload again makes all the
>>> information disappear (including the telephone number I just entered).

Re: [Freeipa-users] be_pam_handler_callback Backend returned: (3, 4, ) [Internal Error (System error)]

[thierry bordaz]

On 01/26/2017 10:55 AM, Harald Dunkel wrote:

Hi Thierry,

good new: I got rid of most of the conflicting entries. There
are only 2 left (see below). They look circular somehow.


That is excellent news. Great !


Please note that the unwanted list of ipa servers is empty. The
official list looks OK. The record for cn=ipaservers,cn=ng,cn=alt\
,dc=example,dc=de looks fine, too. It points to the official list.
So hopefully the duplicates are not a big deal.

It would be nice to get rid of both, though.


Any helpful hint is highly appreciated
Harri
--

% cat <
dn: 
cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de
changetype: delete
EOF

deleting entry 
"cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de"
ldap_delete: Server is unwilling to perform (53)
 additional info: Deleting a managed entry is not allowed. It needs to 
be manually unlinked first.


% cat <
dn: 
cn=ipaservers+nsuniqueid=109be302-ccd911e6-a5b3d0c8-d8da17db,cn=hostgroups,cn=accounts,dc=example,dc=de
changetype: delete
EOF

deleting entry 
"cn=ipaservers+nsuniqueid=109be302-ccd911e6-a5b3d0c8-d8da17db,cn=hostgroups,cn=accounts,dc=example,dc=de"
ldap_delete: Operations error (1)


Those entries are managed entries and it is not possible to delete them 
from direct ldap command.

A solution proposed by Ludwig is not first make them unmanaged:

cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de
changetype: modify
modify: objectclass
delete: mepManagedEntry

cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de
changetype: modify
modify: objectclass
delete: mepManagedEntry

Then retry to delete them.
It should work for the first one but unsure it will succeed for the second one.


% ldapsearch -o ldif-wrap=no -D "cn=directory manager" -w secret -b 
"cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de"
 -s base
# extended LDIF
#
# LDAPv3
# base 

Re: [Freeipa-users] performance scaling of sssd / freeipa

[Sullivan, Daniel [CRI]]

Sumit,

Thank you for detailed and thorough reply; I really appreciate you taking the 
time to get back to me.

What you describe makes sense, and is in-line with my thoughts and observations 
(and from reading Jakubs performance tuning doc).  You articulated this very 
well; thank you.

We had already increased the 389ds thread number from 30 -> 180 and increased 
the number of processor cores to 6 (based on reading 389’s documentation 30 
threads per cores is roughly appropriate).  This actually helped the problem 
significantly.

This morning, we upped that number again to 12 cores and 360 threads for 389 
DS, and this made the situation even better.  The problem has largely gone away 
at this point.

I turned on the ignore_group_members setting long ago, it did not help with 
this particular problem.

I considered enabling enumeration to try and address this issue, for some 
reason I thought that feature wasn’t implemented for AD domains though.  

Anyway, I think we are good for now, at least on this issue, especially thanks 
to your description of what is actually occurring.

Thank you for your time and for your help!

Best,

Dan

> On Jan 26, 2017, at 9:15 AM, Sumit Bose  wrote:
> 
> On Wed, Jan 25, 2017 at 10:58:34PM +, Sullivan, Daniel [CRI] wrote:
>> Hi,
>> 
>> My apologizes for resurrecting this thread.  This issue is still ongoing, at 
>> this point we’ve been looking at it for over a week and now have more than 
>> one staff member analyzing and trying to resolve it on a full time basis.  I 
>> have some more information that I was hoping an a seasoned IPA expert could 
>> take a look at.   At this point I am fairly certain it is a performance 
>> tuning issue in either sssd or FreeIPA on the our domain controllers.  It 
>> looks to me like the main issue is that when looking up the same user across 
>> a large number of nodes in parallel, all of our available ds389 threads get 
>> blocked with '__lll_robust_lock_wait ()’ for operations involving 
>> ipa_extdom_common.c.  This usually occurs on one of our two DCs, but 
>> occasionally on both.   For example, in the attached output, out of 199 
>> threads in the attached output, 179 are in the status __lll_robust_lock_wait 
>> ().  All of the us...@xxx.uchicago.edu in 
>> this attachment are the same user.
>> 
>> Here is more information about this issue (some of it repeated for 
>> convenience):
>> 
>>  1.  We currently have 2 domain controllers.  Each has 6 processor cores and 
>> 180 threads allocated for 389ds.  We have gone through Red Hat’s performance 
>> tuning guide for directory services made what we felt were appropriate 
>> changes, and made additional tuning modifications to get lowered eviction 
>> rates and high cache hit numbers for 389ds.  We have approximately 220 
>> connections to our domain controllers (from "cn=monitor”), depending on the 
>> test I’ve seen as many as 190 connected to a single DC.
>>  2.  We are using an AD domain where all of our users and groups reside.
>>  3.   I induce this by looking up a user (using the id command) on a large 
>> number of nodes (maybe 200) for a user that has never been looked up before, 
>> and is not cached on either the client, or on the DC.
>>  4.   Before I induce the problem, I can lookup entries in LDAP without 
>> delay or problem (i.e. the LDAP server is performant and responsive, I can 
>> inspect cn=monitor or cn=config and get instantaneous results).
>>  5.  When I do induce the issue, the LDAP server basically becomes 
>> unresponsive (which is expected based on the attached output).  Servicing a 
>> query using the ldapsearchtool (for either cn=monitor or cn=config) can take 
>> upwards of 1-2 minutes or longer.  Eventually the LDAP server will 
>> ‘recover’, i.e. I do not typically need to restart IPA services to get this 
>> working again.
>>  6.  After a lookup fails, subsequent parallel lookups succeed and return 
>> the desired record (presumably from the cache).
>>  7.  It appears that these failures are also characterized by a 
>> corresponding "[monitor_hup] (0x0020): Received SIGHUP.”  in the sssd log.
>>  8.  Right before the problem occurs I see a brief spike in CPU utilization 
>> of the ns-slapd process, then the utilization basically drops to 0 once the 
>> threads are blocked in ns-slapd.
>>  9.  Since we are doing computation in our IPA environment, it is important 
>> that we can perform these types of parallel operations against our IPA 
>> environment at the scale we are testing.
>> 
>> I feel like we are either DoSing the LDAP server or the sss_be / sss_nss 
>> processes, although I am not sure.   Right now we are in the process of 
>> deploying an additional domain controller to see if that helps with 
>> distribution of load.  If anybody could provide any sort of information with 
>> respect addressing the issue in the attached trace I would be very grateful.
> 
> I think your observations 

Re: [Freeipa-users] performance scaling of sssd / freeipa

[Sumit Bose]

On Wed, Jan 25, 2017 at 10:58:34PM +, Sullivan, Daniel [CRI] wrote:
> Hi,
> 
> My apologizes for resurrecting this thread.  This issue is still ongoing, at 
> this point we’ve been looking at it for over a week and now have more than 
> one staff member analyzing and trying to resolve it on a full time basis.  I 
> have some more information that I was hoping an a seasoned IPA expert could 
> take a look at.   At this point I am fairly certain it is a performance 
> tuning issue in either sssd or FreeIPA on the our domain controllers.  It 
> looks to me like the main issue is that when looking up the same user across 
> a large number of nodes in parallel, all of our available ds389 threads get 
> blocked with '__lll_robust_lock_wait ()’ for operations involving 
> ipa_extdom_common.c.  This usually occurs on one of our two DCs, but 
> occasionally on both.   For example, in the attached output, out of 199 
> threads in the attached output, 179 are in the status __lll_robust_lock_wait 
> ().  All of the us...@xxx.uchicago.edu in 
> this attachment are the same user.
> 
> Here is more information about this issue (some of it repeated for 
> convenience):
> 
>   1.  We currently have 2 domain controllers.  Each has 6 processor cores and 
> 180 threads allocated for 389ds.  We have gone through Red Hat’s performance 
> tuning guide for directory services made what we felt were appropriate 
> changes, and made additional tuning modifications to get lowered eviction 
> rates and high cache hit numbers for 389ds.  We have approximately 220 
> connections to our domain controllers (from "cn=monitor”), depending on the 
> test I’ve seen as many as 190 connected to a single DC.
>   2.  We are using an AD domain where all of our users and groups reside.
>   3.   I induce this by looking up a user (using the id command) on a large 
> number of nodes (maybe 200) for a user that has never been looked up before, 
> and is not cached on either the client, or on the DC.
>   4.   Before I induce the problem, I can lookup entries in LDAP without 
> delay or problem (i.e. the LDAP server is performant and responsive, I can 
> inspect cn=monitor or cn=config and get instantaneous results).
>   5.  When I do induce the issue, the LDAP server basically becomes 
> unresponsive (which is expected based on the attached output).  Servicing a 
> query using the ldapsearchtool (for either cn=monitor or cn=config) can take 
> upwards of 1-2 minutes or longer.  Eventually the LDAP server will ‘recover’, 
> i.e. I do not typically need to restart IPA services to get this working 
> again.
>   6.  After a lookup fails, subsequent parallel lookups succeed and return 
> the desired record (presumably from the cache).
>   7.  It appears that these failures are also characterized by a 
> corresponding "[monitor_hup] (0x0020): Received SIGHUP.”  in the sssd log.
>   8.  Right before the problem occurs I see a brief spike in CPU utilization 
> of the ns-slapd process, then the utilization basically drops to 0 once the 
> threads are blocked in ns-slapd.
>   9.  Since we are doing computation in our IPA environment, it is important 
> that we can perform these types of parallel operations against our IPA 
> environment at the scale we are testing.
> 
> I feel like we are either DoSing the LDAP server or the sss_be / sss_nss 
> processes, although I am not sure.   Right now we are in the process of 
> deploying an additional domain controller to see if that helps with 
> distribution of load.  If anybody could provide any sort of information with 
> respect addressing the issue in the attached trace I would be very grateful.

I think your observations are due to the fact that SSSD currently
serializes connections from a single process. Your clients will call the
extdom extended LDAP operation on the IPA server to get the information
about the user from the trusted domain. The extdom plugin runs inside of
389ds and each client connection will run in a different thread. To get
the information about the user from the trusted domain the extdom plugin
calls SSSD and here is where the serialization will happen, i.e. all
threads have to wait until the first one will get his results and the
next thread can talk to SSSD.

With an empty cache the initial lookup of a user and all its groups will
take some time and since you used quite a number of clients all 389ds
worker threads will be "busy" waiting to talk to SSSD so that it would
even be hard for other request, even the ones which do not need to talk
to SSSD, to get through because there are no free worker threads.

To improve the situation maybe setting 'ignore_group_members=True' as
described on
https://jhrozek.wordpress.com/2015/08/19/performance-tuning-sssd-for-large-ipa-ad-trust-deployments/
which you already mentioned might help.

Although in general not recommend depending on the size of the trusted
domain (i.e. the number of users and groups in the 

[Freeipa-users] pki status discrepancies

[Jeff Goddard]

Is there a reason the ipactl status command shows pki stopped even though
the systemctl shows it as running? Here is the example output:

[root@id-management-1 log]# systemctl status pki-tomcatd@pki-tomcat
● pki-tomcatd@pki-tomcat.service - PKI Tomcat Server pki-tomcat
   Loaded: loaded (/lib/systemd/system/pki-tomcatd@.service; enabled;
vendor preset: disabled)
   Active: active (running) since Sat 2016-10-01 00:07:50 EDT; 33min ago
  Process: 22425 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
status=0/SUCCESS)
  Process: 22469 ExecStartPre=/usr/bin/pkidaemon start %i (code=exited,
status=0/SUCCESS)
 Main PID: 22582 (java)
   CGroup:
/system.slice/system-pki\x2dtomcatd.slice/pki-tomcatd@pki-tomcat.service
   └─22582 /usr/lib/jvm/jre-1.8.0-openjdk/bin/java
-DRESTEASY_LIB=/usr/share/java/resteasy-base -classpath
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.j...

Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: Oct 01,
2016 12:07:54 AM org.apache.catalina.startup.HostConfig deployDescriptor
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: INFO:
Deployment of configuration descriptor
/etc/pki/pki-tomcat/Catalina/localhost/pki#js.xml has finished in 993 ms
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: Oct 01,
2016 12:07:54 AM org.apache.coyote.AbstractProtocol start
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: INFO:
Starting ProtocolHandler ["http-bio-8080"]
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: Oct 01,
2016 12:07:54 AM org.apache.coyote.AbstractProtocol start
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: INFO:
Starting ProtocolHandler ["http-bio-8443"]
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: Oct 01,
2016 12:07:54 AM org.apache.coyote.AbstractProtocol start
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: INFO:
Starting ProtocolHandler ["ajp-bio-127.0.0.1-8009"]
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: Oct 01,
2016 12:07:54 AM org.apache.catalina.startup.Catalina start
Oct 01 00:07:54 id-management-1.internal.emerlyn.com server[22582]: INFO:
Server startup in 3313 ms
[root@id-management-1 log]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
ipa_memcached Service: RUNNING
httpd Service: RUNNING
pki-tomcatd Service: STOPPED
smb Service: RUNNING
winbind Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@id-management-1 log]#

The system clock has been set to the past in an attempt to renew expired
certificates. I keep getting CA_UNREACHABLE status messages when trying to
renew the certs and I don't know if this is related or not.

Thanks,

Jeff
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] FreeIPA 4.2 CA issues

[Gendy Tartovsky]

Hi Petr,

# getcert list showed that allcertificates are valid for 10 more months.

Server is listening on both ports 389 and 636 and external service are able
to use them.

Also port 8009 is active, I was able to do a telnet on it from localhost.


On Thu, Jan 26, 2017 at 1:31 PM, Petr Vobornik  wrote:

> On 01/25/2017 02:30 PM, Gendy Tartovsky wrote:
> >   Hi,
> >
> > I'm having a PKI-tomcat issue that started after upgrade.
> > My configuration has 4 servers with CA, where servers 2, 3 and 4 are
> replicated
> > from the first one.
> > At first it didn't cause much trouble since all the issue came down to
> > pki-tomcat getting to start about 2 minutes.
> > But it seems that problem is progressed a lot and is causing issues in
> multiple
> > parts of the system.
> >
> > After upgrading FreeIPA from 4.1 to 4.2  ipactl would not on the first
> node
> > start without the --ignore-service-failures.
> >
> >   I found that in the menu Authentication-->Certificates
> >   I have multiple certificates for same hosts in some cases there were
> up to 30
> > duplicates per host and it is unclear what is generating them.
> >
> > Next issue is that if I try to add a new replica with
> ipa-replica-prepare utility
> > I get an error: "Failed to generate certificate"
> >
> > And the last problem I found is that I am unable to restore a backup.
> > The ipa-restore utility is able to unpack the backup but once I try to
> start
> > FreeIPA on a new node
> > the pki-tomcat fails to start. And I see this message in debug:
> >
> > ipa: DEBUG: Waiting for CA to start...
> > ipa: DEBUG: Starting external process
> > ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
> > '--no-check-certificate' 'https://:8443/ca/admin/ca/getStatus'
> > ipa: DEBUG: Process finished, return code=8
> >
> >
> > In the /var/log/dirsrv/slapd-XXX/errors I see a lot of these
> >   NSMMReplicationPlugin - process_postop: Failed to apply update
> > (57c3cc550002000d) error (-1).  Aborting replication
> session(conn=272420 op=6)
> >
> >   but I'm not sure if it is directly related to the problem.
> >
> >   In /var/log/pki/pki-tomcat/ca/debug I see a lot of these messages:
> > Can't create master connection in LdapBoundConnFactory::getConn! Could
> not
> > connect to LDAP server host bos-admin1.hq.datarobot.com
> >  port 636 Error
> netscape.ldap.LDAPException:
> > IO Error creating JSS SSL Socket
> >
> > My guess was that the CA certificate got expired, so I tried to run
> > 'ipa-cacert-manage renew'
> > but it failed with this message:
> >
> > Resubmitting certmonger request '20151222031110' timed out, please check
> the
> > request manually
> >
> >
> > Don't really know what else to try right now.
> >
>
> Could you check:
>
> Is directory server listening on ports 389 and 636?
>
> Is PKI server listening on port 8009 i.e. if you are hitting bug
> https://fedorahosted.org/freeipa/ticket/6575
>
> You can verify if certs are expired by running
>
> # getcert list
>
> And check expiration date.
> --
> Petr Vobornik
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] FreeIPA 4.2 CA issues

[Petr Vobornik]

On 01/25/2017 02:30 PM, Gendy Tartovsky wrote:
>   Hi,
> 
> I'm having a PKI-tomcat issue that started after upgrade.
> My configuration has 4 servers with CA, where servers 2, 3 and 4 are 
> replicated 
> from the first one.
> At first it didn't cause much trouble since all the issue came down to 
> pki-tomcat getting to start about 2 minutes.
> But it seems that problem is progressed a lot and is causing issues in 
> multiple 
> parts of the system.
> 
> After upgrading FreeIPA from 4.1 to 4.2  ipactl would not on the first node 
> start without the --ignore-service-failures.
> 
>   I found that in the menu Authentication-->Certificates
>   I have multiple certificates for same hosts in some cases there were up to 
> 30 
> duplicates per host and it is unclear what is generating them.
> 
> Next issue is that if I try to add a new replica with ipa-replica-prepare 
> utility
> I get an error: "Failed to generate certificate"
> 
> And the last problem I found is that I am unable to restore a backup.
> The ipa-restore utility is able to unpack the backup but once I try to start 
> FreeIPA on a new node
> the pki-tomcat fails to start. And I see this message in debug:
> 
> ipa: DEBUG: Waiting for CA to start...
> ipa: DEBUG: Starting external process
> ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30' 
> '--no-check-certificate' 'https://:8443/ca/admin/ca/getStatus'
> ipa: DEBUG: Process finished, return code=8
> 
> 
> In the /var/log/dirsrv/slapd-XXX/errors I see a lot of these
>   NSMMReplicationPlugin - process_postop: Failed to apply update 
> (57c3cc550002000d) error (-1).  Aborting replication session(conn=272420 
> op=6)
> 
>   but I'm not sure if it is directly related to the problem.
> 
>   In /var/log/pki/pki-tomcat/ca/debug I see a lot of these messages:
> Can't create master connection in LdapBoundConnFactory::getConn! Could not 
> connect to LDAP server host bos-admin1.hq.datarobot.com 
>  port 636 Error 
> netscape.ldap.LDAPException: 
> IO Error creating JSS SSL Socket
> 
> My guess was that the CA certificate got expired, so I tried to run 
> 'ipa-cacert-manage renew'
> but it failed with this message:
> 
> Resubmitting certmonger request '20151222031110' timed out, please check the 
> request manually
> 
> 
> Don't really know what else to try right now.
> 

Could you check:

Is directory server listening on ports 389 and 636?

Is PKI server listening on port 8009 i.e. if you are hitting bug
https://fedorahosted.org/freeipa/ticket/6575

You can verify if certs are expired by running

# getcert list

And check expiration date.
-- 
Petr Vobornik

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] be_pam_handler_callback Backend returned: (3, 4, ) [Internal Error (System error)]

[Harald Dunkel]

Hi Thierry,

good new: I got rid of most of the conflicting entries. There
are only 2 left (see below). They look circular somehow.

Please note that the unwanted list of ipa servers is empty. The
official list looks OK. The record for cn=ipaservers,cn=ng,cn=alt\
,dc=example,dc=de looks fine, too. It points to the official list.
So hopefully the duplicates are not a big deal.

It would be nice to get rid of both, though.


Any helpful hint is highly appreciated
Harri
--

% cat < dn: 
> cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de
> changetype: delete
> EOF
deleting entry 
"cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de"
ldap_delete: Server is unwilling to perform (53)
additional info: Deleting a managed entry is not allowed. It needs to 
be manually unlinked first.


% cat < dn: 
> cn=ipaservers+nsuniqueid=109be302-ccd911e6-a5b3d0c8-d8da17db,cn=hostgroups,cn=accounts,dc=example,dc=de
> changetype: delete
> EOF
deleting entry 
"cn=ipaservers+nsuniqueid=109be302-ccd911e6-a5b3d0c8-d8da17db,cn=hostgroups,cn=accounts,dc=example,dc=de"
ldap_delete: Operations error (1)


% ldapsearch -o ldif-wrap=no -D "cn=directory manager" -w secret -b 
"cn=ipaservers+nsuniqueid=109be304-ccd911e6-a5b3d0c8-d8da17db,cn=ng,cn=alt,dc=example,dc=de"
 -s base
# extended LDIF
#
# LDAPv3
# base