Hi, This is split into two parts.
https://fedorahosted.org/389/ticket/48820 First, is a set of python 3 fixes: https://fedorahosted.org/389/attachment/ticket/48820/0001-Ticket-Fixes-for-python-3.patch Second is the new lib389 api I would like us to move towards. https://fedorahosted.org/389/attachment/ticket/48820/0002-Ticket-48820-Proof-of-concept-of-orm-style-mapping-o.patch This also cleans up and moves some other cli tools around (But I can break that out if we want). I included it because I adapted some of the tools to use the new style. I the best examples are in backend.py, config.py. The parent classes come from _mapped_object.py -- Why would you write this? Lets look at lib389 right now. We have an api that is very inconsistent. Some have get/set, some getProperties/setProperties. Some take a single value, some take a key, some take an array. Some are based on passing an identifer the function to act on, some are just singletons. Each one is attached to DirSrv, and makes a monolithic and huge api. In the end, it looks like lib389 is annoying to work with, and our own team has resorted to calling inst.add_s/modify_s on values even under cn=config: and we have a inst.config.get/set already! -- What does the new api look like? The idea is that all of our configurations in cn=config derive two styles: First, a static once off object, with a known basedn and properties. (ie cn=config) The second is a subtree of objects each with the same attribute types, but that implement many instances. (IE backends) This new way of handling this manages both types. The idea is to share and re-use as much as possible. Lets have a look at config.py. This is now a subclass of DSLdapObject. class Config(DSLdapObject): .... def __init__(self, conn, batch=False): super(Config, self).__init__(instance=conn, batch=batch) self._dn = DN_CONFIG Our super type, defines these methods: def __unicode__(self): def __str__(self): def set(self, key, value): def get(self, key): def remove(self, key): def delete(self): So we can now, just by deriving the type get access to string printing: config = Config(instance) print(config) >>> 'cn=config' We get a set / get on keys config.set('nsslapd-accesslog-level', '1') r = config.get('nsslapd-rootdn') Remove will remove a value for the attr, and delete will delete the object (unless you set self._protected, then it cannot be deleted. This is the default) A future idea is the batch flag. This will make it so that: config.set('k', 'v') config.set('x', 'y') config.commit() <<-- This actually does the ldap mod of k, x at once. This lets us make lots of changes in a more efficient way, and some values need to be updated in sync. And all we had to do was override self._basedn in class Config()! and we picked up so many functions straight out. So lets have a look at the backends now. The backends (note the plural) is derived from DSLdapObjects class Backends(DSLdapObjects): def __init__(self, instance, batch=False): super(Backends, self).__init__(instance=instance, batch=False) self._objectclasses = [BACKEND_OBJECTCLASS_VALUE] self._create_objectclasses = self._objectclasses + ['top', 'extensibleObject' ] self._filterattrs = ['cn', 'nsslapd-suffix', 'nsslapd-directory'] self._basedn = DN_LDBM self._childobject = Backend self._rdn_attribute = 'cn' self._must_attributes = ['nsslapd-suffix', 'cn'] Here we define that our "Backends" all assert certain properties. * They can be found with a certain objectclass * they must be created with a set of classes. * We can uniquely identify them based on the filterattributes. * They are all found under some basedn * They are named off the "cn" attribute * They must contain a cn and a nsslapd-suffix The _childobject type defines what the *single* backend instance is. Because of the inheritence, Backends already gains: def list(self): def get(self, selector): def create(self, rdn=None, properties=None): Additionally, there is an internal method (that will be explained below) def _validate(self, rdn, properties): Just from setting the attributes of the class, we can now list all backends on the system: bes = Backends(inst) print(bes.list()) We can select a backend based on one of the values of a matching attribute in _filterattrs be = bes.get('userRoot') be = bes.get('dc=example,dc=com') be = bes.get('/var/lib/dirsrv/slapd-localhost/db/userRoot') The be instance we get back in the Backend type. This is derive from DSLdapObject: Just like our config. This means it has all the same methods, such as get, set, __unicode__, as our config! We can see how little code it takes to do this: class Backend(DSLdapObject): def __init__(self, instance, dn=None, batch=False): super(Backend, self).__init__(instance, dn, batch) self._naming_attr = 'cn' That is the *entire* definition of Backend. Finally, the true power of DSLdapObjects is when we go to *create* a new instance. Creation of objects is something that in lib389 is hard. We do a lot of validation and checking to be sure of some things. Because we are deriving this type, we can already do a baseline of validation in our creation. The pattern in: bes = Backends(inst) be = bes.create(properties={'nsslapd-suffix': suffix, 'cn': 'userRoot'}) That's it. The reason for the _validate method on Backends is it allows us to hook and do custom validation for the type. In this case, backends can also take the lib389 properties style dictionary, and _validate will re-map the attributes correctly: be = bes.create(properties={'suffix': suffix, 'name': 'userRoot'}) _validate will help us by checking: * Is properties a valid dictionary of types? * Do we have a valid rdn (from self._rdn_attribute) * Do we have all the values of self._must_attributes satisfied? * Is our rdn going to be utf-8 (which python 3 expects?) At first look it seems like it could be a complex api. But you consider the needed work to extend and create say: class RSAEncrption(DSLdapObject): self.__init__(self, instance=None, batch=False): super(RSAEncryption, self).__init__(instance, batch) self._basedn = 'cn=RSA,cn=encryption,cn=config' And that's it. No more having to write modify, add, etc. We can easily, quickly, and confidently map our Directory Server configuration types into lib389. In the future with rest389 this will pay itself off massively, as a consistent, clean, reliable api is going to make creation and deployment of a rest admin console much, much more effecient. -- Isn't this going to end up nearly being a complete rewrite. Yes. But it needs to happen. Lib389 is straining, and hard to edit right now. We should improve this. -- But aren't there risks here of breaking all our tests? Yes. But there is a solution. In backend.py youll note I have: class BackendLegacy(object): This is the *original* Backend type that DirSrv attaches too. It's still accesible: def __add_brookers__(self): ... from lib389.backend import BackendLegacy as Backend This way, we can rename our existing types to <NAME>Legacy, and still use them in tests. I will add a "deprecation" flag to them, and if we decide to accept the new style of api I will begin to not only re-write our existing types, but our tests that rely on them. This way we can stage the transition over time. -- Are there any other wins here? Yes. I basically finished the port of lib389 to python3 in the process of this. -- Does that mean we can merge this without breaking our existing code and tests? Yes it does! -- Sincerely, William Brown Software Engineer Red Hat, Brisbane
signature.asc
Description: This is a digitally signed message part
-- 389-devel mailing list 389-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org
