Re: [Freeipa-devel] [PATCH] Enable filtering search results by member attributes.
On 12/20/2010 11:20 AM, Jan Zelený wrote: Pavel Zunapz...@redhat.com wrote: On 12/08/2010 08:30 PM, Rob Crittenden wrote: Pavel Zůna wrote: On 2010-11-30 04:06, Rob Crittenden wrote: Pavel Zůna wrote: LDAPSearch base class has now the ability to generate additional options for objects with member attributes. These options are used to filter search results - search only for objects without the specified members. Any class that extends LDAPSearch can benefit from this functionality. This patch enables it for the following objects: group, netgroup, rolegroup, hostgroup, taskgroup Example: ipa group-find --no-users=admin Only direct members are taken into account, but if we need indirect members as well - it's not a problem. Ticket #288 Pavel This works as advertised but I wonder what would happen if a huge list of members was passed in to ignore. Is there a limit on the search filter size (remember that the member will be translated into a full dn so will quickly grow in size). Should we impose a cofigurable limit on the # of members to be excluded? Is there a max search filter size and should we check that we haven't exceeded that before doing a search? rob I tried it out with more than a 1000 users and was getting an unwilling to perform error (search filter nested too deep). After a little bit of investigation, I figured the filter was being generated like this: (((!(a=v))(!(a2=v2 We were going deeper with each additional DN! I updated the patch to generate the filter like this instead: (!(|(a=v)(a2=v2))) Tried it again with more than 1000 users (~55Kb) - it worked and wasn't even slow. Updated patch attached. I also had to fix a bug in ldap2 filter generator, as a result this patch depends on my patch number 43. Pavel You'll need to rebase this against master but otherwise ACK. It might be a small optimization to de-dupe the no-users list but it isn't a priority. rob Re-based patch attached. Pavel This hasn't been already pushed and the patch still applies against master. Can someone push it so the ticket can be closed? Jan ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ACK, pushed to master ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] Enable filtering search results by member attributes.
On 12/08/2010 08:30 PM, Rob Crittenden wrote:
Pavel Zůna wrote:
On 2010-11-30 04:06, Rob Crittenden wrote:
Pavel Zůna wrote:
LDAPSearch base class has now the ability to generate additional
options for objects with member attributes. These options are
used to filter search results - search only for objects without
the specified members.
Any class that extends LDAPSearch can benefit from this functionality.
This patch enables it for the following objects:
group, netgroup, rolegroup, hostgroup, taskgroup
Example:
ipa group-find --no-users=admin
Only direct members are taken into account, but if we need indirect
members as well - it's not a problem.
Ticket #288
Pavel
This works as advertised but I wonder what would happen if a huge list
of members was passed in to ignore. Is there a limit on the search
filter size (remember that the member will be translated into a full dn
so will quickly grow in size).
Should we impose a cofigurable limit on the # of members to be excluded?
Is there a max search filter size and should we check that we haven't
exceeded that before doing a search?
rob
I tried it out with more than a 1000 users and was getting an unwilling
to perform error (search filter nested too deep).
After a little bit of investigation, I figured the filter was being
generated like this:
(((!(a=v))(!(a2=v2
We were going deeper with each additional DN!
I updated the patch to generate the filter like this instead:
(!(|(a=v)(a2=v2)))
Tried it again with more than 1000 users (~55Kb) - it worked and wasn't
even slow.
Updated patch attached.
I also had to fix a bug in ldap2 filter generator, as a result this
patch depends on my patch number 43.
Pavel
You'll need to rebase this against master but otherwise ACK.
It might be a small optimization to de-dupe the no-users list but it
isn't a priority.
rob
Re-based patch attached.
Pavel
From 871b9d2b52175a4209ba2d8bdb12fcc019d871e9 Mon Sep 17 00:00:00 2001
From: Pavel Zuna pz...@redhat.com
Date: Thu, 2 Dec 2010 19:24:11 -0500
Subject: [PATCH] Enable filtering search results by member attributes.
LDAPSearch base class has now the ability to generate additional
options for objects with member attributes. These options are
used to filter search results - search only for objects without
the specified members.
Example:
ipa group-find --no-users=admin
Only direct members are taken into account.
Ticket #288
---
ipalib/plugins/baseldap.py | 34 +-
ipalib/plugins/group.py |2 ++
ipalib/plugins/hostgroup.py |2 +-
ipalib/plugins/netgroup.py |1 +
4 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/ipalib/plugins/baseldap.py b/ipalib/plugins/baseldap.py
index 6b7153b..9635f41 100644
--- a/ipalib/plugins/baseldap.py
+++ b/ipalib/plugins/baseldap.py
@@ -1124,6 +1124,9 @@ class LDAPSearch(CallbackInterface, crud.Search):
Retrieve all LDAP entries matching the given criteria.
+member_attributes = []
+member_param_doc = 'exclude %s with member %s (comma-separated list)'
+
takes_options = (
Int('timelimit?',
label=_('Time Limit'),
@@ -1151,6 +1154,33 @@ class LDAPSearch(CallbackInterface, crud.Search):
def get_options(self):
for option in super(LDAPSearch, self).get_options():
yield option
+for attr in self.member_attributes:
+for ldap_obj_name in self.obj.attribute_members[attr]:
+ldap_obj = self.api.Object[ldap_obj_name]
+name = to_cli(ldap_obj_name)
+doc = self.member_param_doc % (
+self.obj.object_name_plural, ldap_obj.object_name_plural
+)
+yield List('no_%s?' % name, cli_name='no_%ss' % name, doc=doc,
+ label=ldap_obj.object_name)
+
+def get_member_filter(self, ldap, **options):
+filter = ''
+for attr in self.member_attributes:
+for ldap_obj_name in self.obj.attribute_members[attr]:
+param_name = 'no_%s' % to_cli(ldap_obj_name)
+if param_name in options:
+dns = []
+ldap_obj = self.api.Object[ldap_obj_name]
+for pkey in options[param_name]:
+dns.append(ldap_obj.get_dn(pkey))
+flt = ldap.make_filter_from_attr(
+attr, dns, ldap.MATCH_NONE
+)
+filter = ldap.combine_filters(
+(filter, flt), ldap.MATCH_ALL
+)
+return filter
has_output_params = global_output_params
@@ -1192,8 +1222,10 @@ class LDAPSearch(CallbackInterface, crud.Search):
search_kw[a] = term
term_filter = ldap.make_filter(search_kw, exact=False)
+member_filter = self.get_member_filter(ldap, **options)
+
filter = ldap.combine_filters(
-(term_filter,
Re: [Freeipa-devel] [PATCH] Enable filtering search results by member attributes.
On 2010-11-30 04:06, Rob Crittenden wrote:
Pavel Zůna wrote:
LDAPSearch base class has now the ability to generate additional
options for objects with member attributes. These options are
used to filter search results - search only for objects without
the specified members.
Any class that extends LDAPSearch can benefit from this functionality.
This patch enables it for the following objects:
group, netgroup, rolegroup, hostgroup, taskgroup
Example:
ipa group-find --no-users=admin
Only direct members are taken into account, but if we need indirect
members as well - it's not a problem.
Ticket #288
Pavel
This works as advertised but I wonder what would happen if a huge list
of members was passed in to ignore. Is there a limit on the search
filter size (remember that the member will be translated into a full dn
so will quickly grow in size).
Should we impose a cofigurable limit on the # of members to be excluded?
Is there a max search filter size and should we check that we haven't
exceeded that before doing a search?
rob
I tried it out with more than a 1000 users and was getting an unwilling
to perform error (search filter nested too deep).
After a little bit of investigation, I figured the filter was being
generated like this:
(((!(a=v))(!(a2=v2
We were going deeper with each additional DN!
I updated the patch to generate the filter like this instead:
(!(|(a=v)(a2=v2)))
Tried it again with more than 1000 users (~55Kb) - it worked and wasn't
even slow.
Updated patch attached.
I also had to fix a bug in ldap2 filter generator, as a result this
patch depends on my patch number 43.
Pavel
From b8c6fa683715d0221b1be33dde6b8065283125d3 Mon Sep 17 00:00:00 2001
From: Pavel Zuna pz...@redhat.com
Date: Mon, 29 Nov 2010 06:44:09 -0500
Subject: [PATCH 2/2] Enable filtering search results by member attributes.
LDAPSearch base class has now the ability to generate additional
options for objects with member attributes. These options are
used to filter search results - search only for objects without
the specified members.
Example:
ipa group-find --no-users=admin
Only direct members are taken into account.
Ticket #288
---
ipalib/plugins/baseldap.py | 34 +-
ipalib/plugins/group.py |2 ++
ipalib/plugins/hostgroup.py |2 +-
ipalib/plugins/netgroup.py |1 +
ipalib/plugins/rolegroup.py |2 +-
ipalib/plugins/taskgroup.py |2 +-
6 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/ipalib/plugins/baseldap.py b/ipalib/plugins/baseldap.py
index a67b84d..ea5454b 100644
--- a/ipalib/plugins/baseldap.py
+++ b/ipalib/plugins/baseldap.py
@@ -1091,6 +1091,9 @@ class LDAPSearch(CallbackInterface, crud.Search):
Retrieve all LDAP entries matching the given criteria.
+member_attributes = []
+member_param_doc = 'exclude %s with member %s (comma-separated list)'
+
takes_options = (
Int('timelimit?',
label=_('Time Limit'),
@@ -1118,6 +1121,33 @@ class LDAPSearch(CallbackInterface, crud.Search):
def get_options(self):
for option in super(LDAPSearch, self).get_options():
yield option
+for attr in self.member_attributes:
+for ldap_obj_name in self.obj.attribute_members[attr]:
+ldap_obj = self.api.Object[ldap_obj_name]
+name = to_cli(ldap_obj_name)
+doc = self.member_param_doc % (
+self.obj.object_name_plural, ldap_obj.object_name_plural
+)
+yield List('no_%s?' % name, cli_name='no_%ss' % name, doc=doc,
+ label=ldap_obj.object_name)
+
+def get_member_filter(self, ldap, **options):
+filter = ''
+for attr in self.member_attributes:
+for ldap_obj_name in self.obj.attribute_members[attr]:
+param_name = 'no_%s' % to_cli(ldap_obj_name)
+if param_name in options:
+dns = []
+ldap_obj = self.api.Object[ldap_obj_name]
+for pkey in options[param_name]:
+dns.append(ldap_obj.get_dn(pkey))
+flt = ldap.make_filter_from_attr(
+attr, dns, ldap.MATCH_NONE
+)
+filter = ldap.combine_filters(
+(filter, flt), ldap.MATCH_ALL
+)
+return filter
has_output_params = global_output_params
@@ -1159,8 +1189,10 @@ class LDAPSearch(CallbackInterface, crud.Search):
search_kw[a] = term
term_filter = ldap.make_filter(search_kw, exact=False)
+member_filter = self.get_member_filter(ldap, **options)
+
filter = ldap.combine_filters(
-(term_filter, attr_filter), rules=ldap.MATCH_ALL
+(term_filter, attr_filter, member_filter), rules=ldap.MATCH_ALL
)
scope = ldap.SCOPE_ONELEVEL
diff --git
Re: [Freeipa-devel] [PATCH] Enable filtering search results by member attributes.
Pavel Zůna wrote: LDAPSearch base class has now the ability to generate additional options for objects with member attributes. These options are used to filter search results - search only for objects without the specified members. Any class that extends LDAPSearch can benefit from this functionality. This patch enables it for the following objects: group, netgroup, rolegroup, hostgroup, taskgroup Example: ipa group-find --no-users=admin Only direct members are taken into account, but if we need indirect members as well - it's not a problem. Ticket #288 Pavel This works as advertised but I wonder what would happen if a huge list of members was passed in to ignore. Is there a limit on the search filter size (remember that the member will be translated into a full dn so will quickly grow in size). Should we impose a cofigurable limit on the # of members to be excluded? Is there a max search filter size and should we check that we haven't exceeded that before doing a search? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
