Re: [Freeipa-devel] Redesigning LDAP code
On 11.1.2013 16:20, Petr Viktorin wrote: On 01/11/2013 03:55 PM, Rob Crittenden wrote: John Dennis wrote: On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. Yes. I looked at the code again and saw the same thing. Fortunately, there's not too much that needs the api object: creating the connection, `get_ipa_config` which shouldn't really be at this level, CrudBackend-specific things, and `normalize_dn` (which I'd really like to remove but it's probably not worth the effort). My working plan now is to have a ipaldap.LDAPBackend base class (please give me a better name), and subclass ldap2 IPAdmin from that. IPAdmin would just add the legacy API which we'll try to move away from; ldap2 would add the api-specific setup and CrudBackend bits (plus its own legacy methods). I would prefer if ldap2 was the base class, but I guess that's just an implementation detail. ipaldap.LDAPClient sounds better? So, the ticket shouldn't really be named installer code should use ldap2 :) Right. We may need to decouple from api a bit. I haven't looked at this for a while but one of the problems is that api locks its values after finalization which can make things a bit inflexible. We use some nasty override code in some place but it isn't something I'd want to see spread further. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] Redesigning LDAP code
We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. -- Petr³ __NOTOC__ = Overview = Ticket [https://fedorahosted.org/freeipa/ticket/2660 #2660] installer code should use ldap2 This is important to do. We really should have just one API and set of classes for dealing with LDAP. For the DN work we had to refactor a fair amount of code in order to force most things to funnel through one common code location. Because ldap is so decentralized and we had so many different APIs, classes, etc it was a large chunk of work and is only partially completed, it got a lot better but it wasn't finished. The primary thing which needs to be resolved is our use of Entity and Entry classes. There never should have been two almost identical classes. One or both of Entity/Entry needs to be removed. As it stands now we have two basic ways we access ldap results. In the installer code it's mostly via Entity/Entry objects. But in the server code (ldpa2) it's done by accessing the data as returned by the python ldap module (e.g. list of (DN, attr_dict) tuples). We need to decide which of the two basic interfaces we're going to use and converge on it. Each approach has merits. But 3 different APIs for interacting with ldap is 2 too many. = Use Cases = N/A = Design= == Entry representation == LDAP entries will be encapsulated in objects. These will perform type checking and validation (ticket #2357). They should grow from ldap2.LDAPEntry (which is currently just a dn, data namedtuple). These objects will behave like a dict of lists: entry[attrname] = [value] attrname in entry del entry[attrname] entry.keys(), .values(), .items() # but NOT `for key in entry`, see below The keys are case-insensitive but case-preserving. We'll use lists for all attributes, even single-valued ones, because single-valuedness can change. QUESTION: Would having entry.dn as an alias for entry['dn'] be useful enough to break the rules? The object should also rembember its original set of attributes, so we don't have to retrieve them from LDAP again when it's updated. == The connection/backend class == We'll use the ldap2 class. The class has some overly specific helper methods like remove_principal_key or modify_password. We shouldn't add new ones, and the existing ones should be moved away eventually. == Backwards compatibility, porting == For compatibility with existing plugins, the LDAPEntry object will unpack to a tuple: dn, entry_attrs = entry (the entry_attrs can be entry itself, so we keep the object's validation powers) I'd rather not do keep indexing (dn = entry[0]; entry_attrs = entry[1]). We'll also temporarily add the legacy Entry interface (toTupleList, toDict, setValues, data, origDataDict, ...) to LDAPEntry. The IPAdmin class will be subclassed from ldap2. All its uses will be gradually converted to only use the ldap2 interface, at which point IPAdmin can be removed. The backwards compatibility stuff should be removed as soon as it's unused. Of course code using the raw python-ldap API will also be converted to ldap2. = Implementation = No additional requirements or changes discovered during the implementation phase. = Feature Managment = N/A = Major configuration options and enablement = N/A = Replication = N/A = Updates and Upgrades = N/A = Dependencies = N/A = External Impact = N/A = Design page authors = ~~~, jdennis ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. -- John Dennis jden...@redhat.com Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
John Dennis wrote: On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. Right. We may need to decouple from api a bit. I haven't looked at this for a while but one of the problems is that api locks its values after finalization which can make things a bit inflexible. We use some nasty override code in some place but it isn't something I'd want to see spread further. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
On 01/11/2013 09:55 AM, Rob Crittenden wrote: John Dennis wrote: On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. Right. We may need to decouple from api a bit. I haven't looked at this for a while but one of the problems is that api locks its values after finalization which can make things a bit inflexible. We use some nasty override code in some place but it isn't something I'd want to see spread further. Ah, object locking, yes I've been bitten by that too. I'm not sure I recall having problems with locked ldap objects but I've certainly have been frustrated with trying to modify other objects which were locked at api creation time. I wonder if the object locking Jason introduced at the early stages of our development is an example of a good idea that's not wonderful in practice. You either have to find the exact moment where an object gets created and update it then which is sometimes awkward or worse impossible or you have to resort to overriding it with the setattr() big hammer. Judging by our use of setattr it's obvious there are numerous places we need to modify a locked object. It's not clear to me if the issues with modifying a locked object are indicative of problems with the locking concept or bad code structure which forces us to violate the locking concept. -- John Dennis jden...@redhat.com Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
On 01/11/2013 03:55 PM, Rob Crittenden wrote: John Dennis wrote: On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. Yes. I looked at the code again and saw the same thing. Fortunately, there's not too much that needs the api object: creating the connection, `get_ipa_config` which shouldn't really be at this level, CrudBackend-specific things, and `normalize_dn` (which I'd really like to remove but it's probably not worth the effort). My working plan now is to have a ipaldap.LDAPBackend base class (please give me a better name), and subclass ldap2 IPAdmin from that. IPAdmin would just add the legacy API which we'll try to move away from; ldap2 would add the api-specific setup and CrudBackend bits (plus its own legacy methods). So, the ticket shouldn't really be named installer code should use ldap2 :) Right. We may need to decouple from api a bit. I haven't looked at this for a while but one of the problems is that api locks its values after finalization which can make things a bit inflexible. We use some nasty override code in some place but it isn't something I'd want to see spread further. -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Redesigning LDAP code
John Dennis wrote: On 01/11/2013 09:55 AM, Rob Crittenden wrote: John Dennis wrote: On 01/11/2013 09:10 AM, Rob Crittenden wrote: Petr Viktorin wrote: We had a small discussion off-list about how we want IPA's LDAP handling to look in the future. To continue the discussion publicly I've summarized the results and added some of my own ideas to a page. John gets credit for the overview (the mistakes WTFs are mine). The text is on http://freeipa.org/page/V3/LDAP_code, and echoed below. IIRC some of the the python-ldap code is used b/c ldap2 may require a configuration to be set up prior to working. That is one of the nice things about the IPAdmin interface, it is much easier to create connections to other hosts. Good point. But I don't believe that issue affects having a common API or a single point where LDAP data flows through. It might mean having more than one initialization method or subclassing. Right. We may need to decouple from api a bit. I haven't looked at this for a while but one of the problems is that api locks its values after finalization which can make things a bit inflexible. We use some nasty override code in some place but it isn't something I'd want to see spread further. Ah, object locking, yes I've been bitten by that too. I'm not sure I recall having problems with locked ldap objects but I've certainly have been frustrated with trying to modify other objects which were locked at api creation time. I wonder if the object locking Jason introduced at the early stages of our development is an example of a good idea that's not wonderful in practice. You either have to find the exact moment where an object gets created and update it then which is sometimes awkward or worse impossible or you have to resort to overriding it with the setattr() big hammer. Judging by our use of setattr it's obvious there are numerous places we need to modify a locked object. It's not clear to me if the issues with modifying a locked object are indicative of problems with the locking concept or bad code structure which forces us to violate the locking concept. The reasoning IIRC was we didn't want a plugin developer mucking with a lot of this for their one plugin as it would affect the entire server. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
