Re: [389-devel] Instance discovery tools in lib389
On 08/12/2015 08:00 PM, William Brown wrote: On Thu, 2015-08-13 at 07:49 +0930, William wrote: I hope this demonstrates how I want to tie together the other contributions I have made into tools, but also making it possible to test other aspects of 389ds. Comments and advice welcome. Hi William, This looks good to me, it's definitely a good start. I see no reason not to push the patch once you think it's ready and polished enough. Don't forget to add the license text to new files. This actually needs to be done for many of the files in lib389 - I'll open a ticket for that Thanks! I think that the main piece of polish is the helper.py library that backs the cli tools to make them a bit easier to write. I'll make this a bit nicer now and send this back in for another review. Thanks for your time, Sincerely, Here is a slightly more polished CliTools helper, and the addition of a more interesting command line tool example of an attribute query tool. You give it an attribute type and it will provide a list of object classes that Must or May take that attribute. I have found this a very useful tool in my work place where we have a complex custom schema, so it's great to know what objectclass to put on objects when you need to put certain attributes on them. This will be very useful. Things like missing objectclasses for memberOf, and such things, always seem to pop up, so this can definitely be used on that front. The biggest outstanding part for me now is how to best format the output of the tool. Should I use the same logging tool as lib389? Or print things out in the current form? Or something else. I think the current form is fine, the lib389 logging is more for how the tool is running (for troubleshooting, etc), and not what the tool should be doing. Does that make sense :-) Perhaps others on this list might have a different opinion. I'll give this some time, and if no one has any concerns/comments I'll push the patch. Thanks, Mark Sincerely, -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] Instance discovery tools in lib389
I hope this demonstrates how I want to tie together the other contributions I have made into tools, but also making it possible to test other aspects of 389ds. Comments and advice welcome. Hi William, This looks good to me, it's definitely a good start. I see no reason not to push the patch once you think it's ready and polished enough. Don't forget to add the license text to new files. This actually needs to be done for many of the files in lib389 - I'll open a ticket for that Thanks! I think that the main piece of polish is the helper.py library that backs the cli tools to make them a bit easier to write. I'll make this a bit nicer now and send this back in for another review. Thanks for your time, Sincerely, -- William will...@blackhats.net.au -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] Instance discovery tools in lib389
Hi William, On 08/06/2015 12:54 AM, William Brown wrote: Hi, I have been working on lib389 recently to bring in a number of the tools I have developed professionally into the project so that others can benefit from this. As you may have seen I've already submitted a few patches related to this. I've done some of the api changes needed, but now I would like to make it a bit easier to add command line tools that can utilise lib389. This is right inline with what we would like to do as well. I would like to add a tools directory to lib389 with a number of the cli tools and helpers that I have written [1]. Is this the best location for these tools? If there are really simple, maybe they could just be added to the tools.py/utils.py files? More on this below... Some of them could arguably go into the main line 389ds codebase, but I want to use lib389 to back them. This really depends on the tool, but see below... Second, with these tools I don't want them to be complex. +1 For lib389 inclusion they should be very simple, so that other tools can utilize them. For example, I would like a tool that can show what attribute can be held by what objectClass. I would want the calling syntax to be as easy as: attribute_query.py -i instance name attribute name In order to achieve this there are a few changes I want to make. First, is that there is already a DirSrv.list() function. This works well to list instances on the system, but it's missing the instance name, so I want to add this to the dictionary that is returned. Second, I want to make it possible to initialise the DirSrv object from an instance directory returned by the list() function. Normally this is done through the allocate() function to establish the details of the connection, at the moment the instance function doesn't return enough data to support this. Either, I can add in functionality to the list() function to return more data about each instance, possibly enough to just take the result from list() and put it in as allocate(args) and have a working connection. Alternately I write a new allocate_from_instance() (or some other name ...?) function that is able to do the discovery of the needed data, and then call allocate. Which would be the preferred method. Both. Expand the instance dictionary(your proposal could be useful for many other tasks/tools), and create the new allocate_from_instance function. Comments and advice is appreciated. [1] The tools I want to add are: * List all known system instances * schema query tools (list of object classes, attributes, query which objectclasses can take which attribute) This should really be added to schema.py, or at least the core functionality added to schema.py, then build your tool off of that? * aci testing and manipulation tools. For example, does aci X do what I suspect. * aci inversion tool. convert != aci's into == aci's to help conversion to white lists. Maybe these aci tools could be partially integrated into a new acl.py file? Kind of like what I suggested with the schema tools. * Probably many more that I haven't thought of yet to help make administration of a 389ds instance simpler. Ultimately, I would like to see some of your new functionally go into the existing core module, and then write your tools based off of those. This way it can be leveraged outside of your specific task tools - for use in other new/future tools. Of course I am specualting a bit, because I have not seen what you are working on yet. Perhaps others on this list might have some opinions thoughts as well? In the meantime I would like to see what you are working on, and we can discuss this together and work towards an ideal approach. Thanks again for your interest and contribution to the lib389 project! Mark -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] Instance discovery tools in lib389
I've done some of the api changes needed, but now I would like to make it a bit easier to add command line tools that can utilise lib389. This is right inline with what we would like to do as well. Glad to hear it. I would like to add a tools directory to lib389 with a number of the cli tools and helpers that I have written [1]. Is this the best location for these tools? If there are really simple, maybe they could just be added to the tools.py/utils.py files? More on this below... Yes and no. See below, as I think I explain a bit better what I'm working towards. Some of them could arguably go into the main line 389ds codebase, but I want to use lib389 to back them. This really depends on the tool, but see below... Second, with these tools I don't want them to be complex. +1 For lib389 inclusion they should be very simple, so that other tools can utilize them. Glad that we agree on this. Either, I can add in functionality to the list() function to return more data about each instance, possibly enough to just take the result from list() and put it in as allocate(args) and have a working connection. Alternately I write a new allocate_from_instance() (or some other name ...?) function that is able to do the discovery of the needed data, and then call allocate. Which would be the preferred method. Both. Expand the instance dictionary(your proposal could be useful for many other tasks/tools), and create the new allocate_from_instance function. Well, there is no point doing both. I was either going to expand the instance dict to include all the SER_* attributes, so that you could literally pass it straight to allocate(). Or, you have just the instance name added to the instance dict, then you have allocate_from_instance (Which also, I don't like that name even though I came up with it. It's confusing :S ) which will then discover all the SER_* values, then pass them all to allocate() anyway. This is why I think we should choose on way or the other. My preference is actually the former, as we avoid a confusing function name, and it makes the value of the list() function greater. You could do: insts = ds.list(all=True) someinst = insts[0] ds.allocate(someinst) Or even allocate a new DirSrv, etc. Comments and advice is appreciated. [1] The tools I want to add are: * List all known system instances * schema query tools (list of object classes, attributes, query which objectclasses can take which attribute) This should really be added to schema.py, or at least the core functionality added to schema.py, then build your tool off of that? * aci testing and manipulation tools. For example, does aci X do what I suspect. * aci inversion tool. convert != aci's into == aci's to help conversion to white lists. Maybe these aci tools could be partially integrated into a new acl.py file? Kind of like what I suggested with the schema tools. * Probably many more that I haven't thought of yet to help make administration of a 389ds instance simpler. Ultimately, I would like to see some of your new functionally go into the existing core module, and then write your tools based off of those. This way it can be leveraged outside of your specific task tools - for use in other new/future tools. Of course I am specualting a bit, because I have not seen what you are working on yet. You have already seen my work, and acked most of the functionality into the core modules. Additionally the aci.py module is waiting in your inbox for informal review. For example: https://fedorahosted.org/389/ticket/48236 - Will be used in conjunction with aci.py to evaluate and test aci functionality is working as expected, both in the aci evaluation engine in 389ds, but also from a semantic point of view of user acis https://fedorahosted.org/389/ticket/48238 - Is all the functionality needed to make the schema query tool I was discussing before. My goal was to always add functionality into the library, then make the most minimal command line tool possible to just stich it all together. This way both unit tests and possible cli tools can benefit from the work. So I think we already have the same approach, I just maybe didn't communicate that very well. Perhaps others on this list might have some opinions thoughts as well? In the meantime I would like to see what you are working on, and we can discuss this together and work towards an ideal approach. Sounds like a plan! Thanks again for your interest and contribution to the lib389 project! You're welcome. Thanks for your time reviewing my work and ideas. -- William Brown will...@blackhats.net.au -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
