Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-05-13 Thread Rob Crittenden

Martin Basti wrote:



On 12.05.2016 19:48, Rob Crittenden wrote:

Martin Basti wrote:



On 22.04.2016 13:21, David Kupka wrote:

On 22/04/16 10:58, Martin Basti wrote:



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.




Works for me, ACK.


pushed to master:
* fe2ce02a6f7664e377c367e16e9c2e1ad960c9d7 Performace: don't download
password attributes in host/user-find



It occurs to me, won't this break the UI somewhat. Isn't Enrolled one
of the attributes on the default host page. Won't this show all hosts
as unenrolled?

rob


Hi Rob,

how exactly is webUI broken? I tried host section and it works. IIUC the
Web UI uses just host-find --pkey-only, which is not affected by this
change. For additional details webUI call host-show for each listed entry


You're right, it was based on a bad assumption on my part. I assumed the 
UI took the output of host_find and displayed the results. Next time 
I'll check first.


rob

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-05-13 Thread Martin Basti



On 12.05.2016 19:48, Rob Crittenden wrote:

Martin Basti wrote:



On 22.04.2016 13:21, David Kupka wrote:

On 22/04/16 10:58, Martin Basti wrote:



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.




Works for me, ACK.


pushed to master:
* fe2ce02a6f7664e377c367e16e9c2e1ad960c9d7 Performace: don't download
password attributes in host/user-find



It occurs to me, won't this break the UI somewhat. Isn't Enrolled one 
of the attributes on the default host page. Won't this show all hosts 
as unenrolled?


rob


Hi Rob,

how exactly is webUI broken? I tried host section and it works. IIUC the 
Web UI uses just host-find --pkey-only, which is not affected by this 
change. For additional details webUI call host-show for each listed entry


Martin

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-05-12 Thread Rob Crittenden

Martin Basti wrote:



On 22.04.2016 13:21, David Kupka wrote:

On 22/04/16 10:58, Martin Basti wrote:



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.




Works for me, ACK.


pushed to master:
* fe2ce02a6f7664e377c367e16e9c2e1ad960c9d7 Performace: don't download
password attributes in host/user-find



It occurs to me, won't this break the UI somewhat. Isn't Enrolled one of 
the attributes on the default host page. Won't this show all hosts as 
unenrolled?


rob

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-04-26 Thread Martin Basti



On 22.04.2016 13:21, David Kupka wrote:

On 22/04/16 10:58, Martin Basti wrote:



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.




Works for me, ACK.


pushed to master:
* fe2ce02a6f7664e377c367e16e9c2e1ad960c9d7 Performace: don't download 
password attributes in host/user-find


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-04-22 Thread David Kupka

On 22/04/16 10:58, Martin Basti wrote:



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.




Works for me, ACK.

--
David Kupka

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-04-22 Thread Martin Basti



On 21.04.2016 09:17, Martin Basti wrote:



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK



Updated patch attached.
From bc4c2a0b206b8b85c6c66088575efedc772e5a21 Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Fri, 8 Apr 2016 16:18:08 +0200
Subject: [PATCH] Performace: don't download password attributes in
 host/user-find

For each entry in user/host-find was executed an extra search for password
attributes what has significant impact on performance (for 2000 users
there were 2000 additional searches)

http://www.freeipa.org/page/V4/Performance_Improvements

https://fedorahosted.org/freeipa/ticket/5281
---
 ipalib/plugins/baseuser.py  | 1 -
 ipalib/plugins/host.py  | 5 -
 ipatests/test_xmlrpc/test_host_plugin.py| 2 +-
 ipatests/test_xmlrpc/tracker/host_plugin.py | 6 --
 ipatests/test_xmlrpc/tracker/user_plugin.py | 8 ++--
 5 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/ipalib/plugins/baseuser.py b/ipalib/plugins/baseuser.py
index 252d40ae3828417d9692510d5036aaadaeb9edce..cb6bf263160d33d396d740d6e25be811cf816c71 100644
--- a/ipalib/plugins/baseuser.py
+++ b/ipalib/plugins/baseuser.py
@@ -632,7 +632,6 @@ class baseuser_find(LDAPSearch):
 
 def post_common_callback(self, ldap, entries, lockout=False, **options):
 for attrs in entries:
-self.obj.get_password_attributes(ldap, attrs.dn, attrs)
 self.obj.convert_usercertificate_post(attrs, **options)
 if (lockout):
 attrs['nsaccountlock'] = True
diff --git a/ipalib/plugins/host.py b/ipalib/plugins/host.py
index 19327d8323d945062e06ccdb33ea2106cd1c6a00..611f9a9c7c51d5f81ba9ca91c6a900028aa249f4 100644
--- a/ipalib/plugins/host.py
+++ b/ipalib/plugins/host.py
@@ -1032,12 +1032,7 @@ class host_find(LDAPSearch):
 set_certificate_attrs(entry_attrs)
 set_kerberos_attrs(entry_attrs, options)
 rename_ipaallowedtoperform_from_ldap(entry_attrs, options)
-self.obj.get_password_attributes(ldap, entry_attrs.dn, entry_attrs)
 self.obj.suppress_netgroup_memberof(ldap, entry_attrs)
-if entry_attrs['has_password']:
-# If an OTP is set there is no keytab, at least not one
-# fetched anywhere.
-entry_attrs['has_keytab'] = False
 
 if options.get('all', False):
 entry_attrs['managing'] = self.obj.get_managed_hosts(entry_attrs.dn)
diff --git a/ipatests/test_xmlrpc/test_host_plugin.py b/ipatests/test_xmlrpc/test_host_plugin.py
index 47f05a403ddb519f201b11251c2acb71faa9133b..ea8f49656b47b91950130f78375f1c4447157ecb 100644
--- a/ipatests/test_xmlrpc/test_host_plugin.py
+++ b/ipatests/test_xmlrpc/test_host_plugin.py
@@ -416,7 +416,7 @@ class TestManagedHosts(XMLRPC_test):
 count=1,
 truncated=False,
 summary=u'1 host matched',
-result=[host.filter_attrs(host.retrieve_keys)],
+result=[host.filter_attrs(host.find_keys)],
 ), result)
 
 def search_man_hosts(self, host1, host2):
diff --git a/ipatests/test_xmlrpc/tracker/host_plugin.py b/ipatests/test_xmlrpc/tracker/host_plugin.py
index 0a69d39c0b77f3f6a7f09ef0785a78143b586c8f..67faa1acf9eeb6174e8d09ca012ae20636fb0f51 100644
--- a/ipatests/test_xmlrpc/tracker/host_plugin.py
+++ b/ipatests/test_xmlrpc/tracker/host_plugin.py
@@ -42,6 +42,8 @@ class HostTracker(Tracker):
 update_keys = retrieve_keys - {'dn'}
 managedby_keys = retrieve_keys - {'has_keytab', 'has_password'}
 allowedto_keys = retrieve_keys - {'has_keytab', 'has_password'}
+find_keys = retrieve_keys - {'has_keytab', 'has_password'}
+find_all_keys = retrieve_all_keys - {'has_keytab', 'has_password'}
 
 def __init__(self, name, fqdn=None, default_version=None):
 super(HostTracker, self).__init__(default_version=default_version)
@@ -136,9 +138,9 @@ class HostTracker(Tracker):
 def check_find(self, result, all=False, raw=False):
 """Check `host_find` command result"""
 if all:
-expected = self.filter_attrs(self.retrieve_all_keys)
+expected = self.filter_attrs(self.find_all_keys)
 else:
-expected = self.filter_attrs(self.retrieve_keys)
+expected = self.filter_attrs(self.find_keys)
 assert_deepequal(dict(
 count=1,
 truncated=False,
diff --git a/ipatests/test_xmlrpc/tracker/user_plugin.py b/ipatests/test_xmlrpc/tracker/user_plugin.py
index 216112db50ff909846e7bb900e76e961177cd10b..5acfc63cd62e28c7dd3ce1060bbf4fd262920f22 100644
--- a/ipatests/test_xmlrpc/tracker/user_plugin.py
+++ b/ipatests/test_xmlrpc/tracker/user_plugin.py
@@ -51,8 +51,12 @@ class UserTracker(Tracker):
 update_keys = retrieve_keys - {u'dn'}
 activate_keys = retrieve_keys
 
-find_keys = retrieve_keys - {u'mepmanagedentry', 

Re: [Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-04-21 Thread Martin Basti



On 20.04.2016 16:57, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.



selfNACK
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0463] Performance: do not download password attributes in host/find-user command

2016-04-20 Thread Martin Basti

https://fedorahosted.org/freeipa/ticket/5281

Patch attached.
From 96dcfec0399ccefbfcaebf2b7d19ea680aedeffa Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Fri, 8 Apr 2016 16:18:08 +0200
Subject: [PATCH] Performace: don't download password attributes in
 host/user-find

For each entry in user/host-find was executed an extra search for password
attributes what has significant impact on performance (for 2000 users
there were 2000 additional searches)

http://www.freeipa.org/page/V4/Performance_Improvements

https://fedorahosted.org/freeipa/ticket/5281
---
 ipalib/plugins/baseuser.py | 1 -
 ipalib/plugins/host.py | 1 -
 2 files changed, 2 deletions(-)

diff --git a/ipalib/plugins/baseuser.py b/ipalib/plugins/baseuser.py
index 252d40ae3828417d9692510d5036aaadaeb9edce..cb6bf263160d33d396d740d6e25be811cf816c71 100644
--- a/ipalib/plugins/baseuser.py
+++ b/ipalib/plugins/baseuser.py
@@ -632,7 +632,6 @@ class baseuser_find(LDAPSearch):
 
 def post_common_callback(self, ldap, entries, lockout=False, **options):
 for attrs in entries:
-self.obj.get_password_attributes(ldap, attrs.dn, attrs)
 self.obj.convert_usercertificate_post(attrs, **options)
 if (lockout):
 attrs['nsaccountlock'] = True
diff --git a/ipalib/plugins/host.py b/ipalib/plugins/host.py
index 19327d8323d945062e06ccdb33ea2106cd1c6a00..a7a6048e7d8fcebc0a291083893ac971ec94f25f 100644
--- a/ipalib/plugins/host.py
+++ b/ipalib/plugins/host.py
@@ -1032,7 +1032,6 @@ class host_find(LDAPSearch):
 set_certificate_attrs(entry_attrs)
 set_kerberos_attrs(entry_attrs, options)
 rename_ipaallowedtoperform_from_ldap(entry_attrs, options)
-self.obj.get_password_attributes(ldap, entry_attrs.dn, entry_attrs)
 self.obj.suppress_netgroup_memberof(ldap, entry_attrs)
 if entry_attrs['has_password']:
 # If an OTP is set there is no keytab, at least not one
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code