Re: [Freeipa-users] Slow user logon with IPA
On Wed, Apr 22, 2015 at 12:43:47AM +0200, Mateusz Malek wrote: On 15.04.2015 at 15:08, Lukas Slebodnik wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) Packages for fedora 21,22 are built. You just need to wait utill they are available in updates testing or you can download packages from koji. https://admin.fedoraproject.org/updates/sssd-1.12.4-4.fc22 https://admin.fedoraproject.org/updates/sssd-1.12.4-3.fc21 Please test and provide karma. Well, it took me a long time, but I can confirm that with these packages logon times seem fine. Thank you all for quick help! Now I'm only waiting for updated packages to apper in CentOS/RHEL repositories; in my test environment I'm perfectly fine with backporting them on my own. Currently the fix is scheduled to appear in 7.2 only. If you need the fix sooner, then we need a support case open.. The fix is also present in 6.7 to avoid regressing compared to 6.6 -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On 14.04.2015 at 21:30, Rich Megginson wrote: On 04/14/2015 12:35 PM, thierry bordaz wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) When such long requests happened, you may take several pstack of the 389-ds process. Ideally you can timestamp the pstack output so that it is easier to correlate with DS access logs. Providing pstacks+access/errors logs would really help to know if there is a bottleneck. See also http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs You'll need to do debuginfo-install ipa-server slapi-nis I've tried looking into captured information, but I think that there's nothing suspicious. With selinux_provider patched speed is pretty good - FreeIPA has more to do during user logon than our existing setup had (obtaining Kerberos ticket and processing HBAC rules is definitely more complex than single lookup with pam_ldap/nss_ldap) and I'll probably blame those longer LDAP search times (that happen from time to time) on our datastore performance. Thank you all, again. Best regards Mateusz Małek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On 15.04.2015 at 15:08, Lukas Slebodnik wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) Packages for fedora 21,22 are built. You just need to wait utill they are available in updates testing or you can download packages from koji. https://admin.fedoraproject.org/updates/sssd-1.12.4-4.fc22 https://admin.fedoraproject.org/updates/sssd-1.12.4-3.fc21 Please test and provide karma. Well, it took me a long time, but I can confirm that with these packages logon times seem fine. Thank you all for quick help! Now I'm only waiting for updated packages to apper in CentOS/RHEL repositories; in my test environment I'm perfectly fine with backporting them on my own. Best regards Mateusz Małek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
2015-04-15 15:08 GMT+02:00 Lukas Slebodnik lsleb...@redhat.com: On (15/04/15 08:53), Jakub Hrozek wrote: I pushed the selinux performance patches upstream yesterday. They will make their way to 7.2, 6.7 and I guess Lukas might also cherry-pick them for Fedora. Packages for fedora 21,22 are built. You just need to wait utill they are available in updates testing or you can download packages from koji. https://admin.fedoraproject.org/updates/sssd-1.12.4-4.fc22 https://admin.fedoraproject.org/updates/sssd-1.12.4-3.fc21 Please test and provide karma. Karma provided. For my setup I'm finally back to the 3-4 seconds login time for a user with only a handful of groups. Thanks! -- john -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On (15/04/15 08:53), Jakub Hrozek wrote: On Tue, Apr 14, 2015 at 05:36:16PM +0200, Mateusz Malek wrote: On Fri, Apr 10, 2015 at 08:48 PM, Jakub Hrozek wrote: On Fri, Apr 10, 2015 at 12:39:20PM -0400, Dmitri Pal wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) (...) Do authentication and see where the time is spent by examining the logs. Correlate it to the logs on the server. (...) I spent the better part of today fixing this issue: https://fedorahosted.org/sssd/ticket/2624 You might want to check if you're hit by this bug by setting: selinux_provider=none temporarily. With selinux_provider=none things seems faster. It's still not as fast as with existing OpenLDAP, but logon times seem acceptable now (they mostly vary from 0.5 to 2 seconds, sometimes they go up to 3 seconds). It seems that most time is spent in Kerberos authentication (logs just stop flowing for a while) and on HBAC processing - on the 389 DS side it seems that LDAP is busy with requests (it looks like it sometimes hangs on MOD operation - is it updating user last logon time?). I pushed the selinux performance patches upstream yesterday. They will make their way to 7.2, 6.7 and I guess Lukas might also cherry-pick them for Fedora. Packages for fedora 21,22 are built. You just need to wait utill they are available in updates testing or you can download packages from koji. https://admin.fedoraproject.org/updates/sssd-1.12.4-4.fc22 https://admin.fedoraproject.org/updates/sssd-1.12.4-3.fc21 Please test and provide karma. LS -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On Tue, Apr 14, 2015 at 05:36:16PM +0200, Mateusz Malek wrote: On Fri, Apr 10, 2015 at 08:48 PM, Jakub Hrozek wrote: On Fri, Apr 10, 2015 at 12:39:20PM -0400, Dmitri Pal wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) (...) Do authentication and see where the time is spent by examining the logs. Correlate it to the logs on the server. (...) I spent the better part of today fixing this issue: https://fedorahosted.org/sssd/ticket/2624 You might want to check if you're hit by this bug by setting: selinux_provider=none temporarily. With selinux_provider=none things seems faster. It's still not as fast as with existing OpenLDAP, but logon times seem acceptable now (they mostly vary from 0.5 to 2 seconds, sometimes they go up to 3 seconds). It seems that most time is spent in Kerberos authentication (logs just stop flowing for a while) and on HBAC processing - on the 389 DS side it seems that LDAP is busy with requests (it looks like it sometimes hangs on MOD operation - is it updating user last logon time?). I pushed the selinux performance patches upstream yesterday. They will make their way to 7.2, 6.7 and I guess Lukas might also cherry-pick them for Fedora. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On 04/14/2015 05:36 PM, Mateusz Malek wrote: On Fri, Apr 10, 2015 at 08:48 PM, Jakub Hrozek wrote: On Fri, Apr 10, 2015 at 12:39:20PM -0400, Dmitri Pal wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) (...) Do authentication and see where the time is spent by examining the logs. Correlate it to the logs on the server. (...) I spent the better part of today fixing this issue: https://fedorahosted.org/sssd/ticket/2624 You might want to check if you're hit by this bug by setting: selinux_provider=none temporarily. With selinux_provider=none things seems faster. It's still not as fast as with existing OpenLDAP, but logon times seem acceptable now (they mostly vary from 0.5 to 2 seconds, sometimes they go up to 3 seconds). It seems that most time is spent in Kerberos authentication (logs just stop flowing for a while) and on HBAC processing - on the 389 DS side it seems that LDAP is busy with requests (it looks like it sometimes hangs on MOD operation - is it updating user last logon time?). Hello, When such long requests happened, you may take several pstack of the 389-ds process. Ideally you can timestamp the pstack output so that it is easier to correlate with DS access logs. Providing pstacks+access/errors logs would really help to know if there is a bottleneck. thanks Best regards, Mateusz Malek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On Fri, Apr 10, 2015 at 08:48 PM, Jakub Hrozek wrote: On Fri, Apr 10, 2015 at 12:39:20PM -0400, Dmitri Pal wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) (...) Do authentication and see where the time is spent by examining the logs. Correlate it to the logs on the server. (...) I spent the better part of today fixing this issue: https://fedorahosted.org/sssd/ticket/2624 You might want to check if you're hit by this bug by setting: selinux_provider=none temporarily. With selinux_provider=none things seems faster. It's still not as fast as with existing OpenLDAP, but logon times seem acceptable now (they mostly vary from 0.5 to 2 seconds, sometimes they go up to 3 seconds). It seems that most time is spent in Kerberos authentication (logs just stop flowing for a while) and on HBAC processing - on the 389 DS side it seems that LDAP is busy with requests (it looks like it sometimes hangs on MOD operation - is it updating user last logon time?). Best regards, Mateusz Malek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] Slow user logon with IPA
On 04/14/2015 12:35 PM, thierry bordaz wrote: On 04/14/2015 05:36 PM, Mateusz Malek wrote: On Fri, Apr 10, 2015 at 08:48 PM, Jakub Hrozek wrote: On Fri, Apr 10, 2015 at 12:39:20PM -0400, Dmitri Pal wrote: On 04/10/2015 08:13 AM, Mateusz Malek wrote: I'm about to migrate my OpenLDAP-based environment to FreeIPA, however I've hit some weird performance problems. When I'm using IPA, it takes about 5-7 (or even more) seconds to get shell prompt after entering user password (...) (...) Do authentication and see where the time is spent by examining the logs. Correlate it to the logs on the server. (...) I spent the better part of today fixing this issue: https://fedorahosted.org/sssd/ticket/2624 You might want to check if you're hit by this bug by setting: selinux_provider=none temporarily. With selinux_provider=none things seems faster. It's still not as fast as with existing OpenLDAP, but logon times seem acceptable now (they mostly vary from 0.5 to 2 seconds, sometimes they go up to 3 seconds). It seems that most time is spent in Kerberos authentication (logs just stop flowing for a while) and on HBAC processing - on the 389 DS side it seems that LDAP is busy with requests (it looks like it sometimes hangs on MOD operation - is it updating user last logon time?). Hello, When such long requests happened, you may take several pstack of the 389-ds process. Ideally you can timestamp the pstack output so that it is easier to correlate with DS access logs. Providing pstacks+access/errors logs would really help to know if there is a bottleneck. See also http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs You'll need to do debuginfo-install ipa-server slapi-nis thanks Best regards, Mateusz Malek -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
