On 20.7.2016 19:25, Ben Lipton wrote: > On 07/20/2016 12:21 PM, Simo Sorce wrote: >> On Wed, 2016-07-20 at 12:14 -0400, Ben Lipton wrote: >>> On 07/20/2016 10:37 AM, Simo Sorce wrote: >>>> On Wed, 2016-07-20 at 10:17 -0400, Ben Lipton wrote: >>>>> On 07/20/2016 06:27 AM, Simo Sorce wrote: >>>>>> On Tue, 2016-07-19 at 16:20 -0400, Ben Lipton wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I have updated the design page >>>>>>> http://www.freeipa.org/page/V4/Automatic_Certificate_Request_ >>>>>>> Gene >>>>>>> rati >>>>>>> on/Mapping_Rules >>>>>>> with my plan for implementing user-configurable rules for >>>>>>> mapping >>>>>>> IPA >>>>>>> data into certificate requests. In brief: we will use Jinja2 >>>>>>> for >>>>>>> templating. Data rules (which map individual data items) and >>>>>>> syntax >>>>>>> rules (which group them into certificate fields) will both be >>>>>>> snippets >>>>>>> of Jinja2 markup. The formatting process will be as follows:
I've finally found some time to read the sub-page Mapping_Rules and for me it is kind of hard to follow. It would not be understandable without this e-mail and the blog posts (BTW the blog articles are among best I have seen!). Most importantly, the explanations in brackets above ["Data rules (which map individual data items) and (which group them into certificate fields)"] are missing in the wiki page itself :-) Could you fold relevant parts of the e-mails and blogs back into the wiki page so it is self-contained? Besides this nit, http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Mapping_Rules#Planned_implementation sounds reasonable. I like how it prevents bad data from template-injection. Regarding http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A , I prefer Option A with separate object for each helper. It is somehow cleaner and it might be useful to use distinct object classes for each helper etc. API for ipa cert-get-requestdata sounds good. API for ipa cert-request makes sense to me as well. In any case I would recommend you to consult API design with Jan Cholasta <jchol...@redhat.com> - he is our API custodian. BTW I very much like "Alternatives considered" sections, we should have this for each design! Good work, I really like the dutiful analysis! >>>>>>> 1. Syntax rules will be rendered using Jinja2. Data rules >>>>>>> (rule >>>>>>> text, >>>>>>> not rendered) will be passed as the datarules attribute. >>>>>>> 2. Rendered syntax rules will be processed by the Formatter >>>>>>> class >>>>>>> for >>>>>>> the selected CSR generation helper (e.g. openssl or >>>>>>> certutil). >>>>>>> The >>>>>>> formatter combines these partial rules into a full template >>>>>>> for >>>>>>> the >>>>>>> config. >>>>>>> 3. The template will be rendered using Jinja2. Relevant data >>>>>>> from >>>>>>> the >>>>>>> IPA database will be available in the context for this >>>>>>> rendering. >>>>>>> 4. The final rendered template will be returned to the >>>>>>> caller, >>>>>>> labeled >>>>>>> with its function (e.g. a command line or a config file). >>>>>>> >>>>>>> Are there any comments or objections to this approach? Here's >>>>>>> an >>>>>>> example >>>>>>> to show what it might look like in practice. >>>>>>> >>>>>>> Example data rules: >>>>>>> email={{subject.email}} >>>>>>> O={{config.ipacertificatesubjectbase}}\nCN={{subject.username >>>>>>> }} >>>>>>> >>>>>>> Example syntax rule: >>>>>>> subjectAltName=@{% section %}{{datarules|join('\n')}}{% >>>>>>> endsection %} >>>>>>> >>>>>>> Example composed config template: >>>>>>> [ req ] >>>>>>> prompt = no >>>>>>> encrypt_key = no >>>>>>> >>>>>>> distinguished_name = {% section >>>>>>> %}O={{config.ipacertificatesubjectbase}} >>>>>>> CN={{subject.username}}{% endsection %} >>>>>>> >>>>>>> req_extensions = exts >>>>>>> >>>>>>> [ exts ] >>>>>>> subjectAltName=@{% section %}email={{subject.email}}{% >>>>>>> endsection >>>>>>> %} >>>>>>> >>>>>>> There's a lot more information about the thinking behind this >>>>>>> at >>>>>>> http://blog.benjaminlipton.com/2016/07/19/csr-generation-temp >>>>>>> lati >>>>>>> ng.h >>>>>>> tml >>>>>>> if you're interested, as well. >>>>>> Nice work Ben, >>>>>> it's been really nice to be able to follow your notes on the >>>>>> blog >>>>>> post, >>>>>> one question remains lingering in my head, why jinja2 ? >>>>>> I know that engine relatively well as I used it in ipsilon, so >>>>>> I am >>>>>> not >>>>>> questioning the choice just asking why specifically jinja2 and >>>>>> not >>>>>> something else, potentially language agnostic. >>>>>> >>>>>> Simo. >>>>> Honestly, my reasoning didn't go very far beyond that it seems to >>>>> be >>>>> widely used and is compatible with python, which is the language >>>>> where >>>>> the implementation is taking place (in the IPA RPC server). I >>>>> thought >>>>> about using the built-in python format strings or creating a >>>>> simple >>>>> domain-specific language, but the likelihood of wanting the >>>>> built-in >>>>> text processing features (join, replace, maybe even for loops) >>>>> seemed >>>>> high, and I didn't want to reimplement those features. >>>>> >>>>> Will the additional package dependency be a problem? >>>> I am more concerned a out the ability to process the data (which I >>>> guess is stored in LDAP) by another client, or in the CLI. >>>> Other than that the dependency does not concern me too much >>>> provided >>>> jinja2 templating is stable and has some guarantee that it will be >>>> supportable long term. >>>> >>>> If that is not guaranteed it is a problem, we cannot easily swap >>>> out >>>> one language for another once data is stored and used by the >>>> server. >>>> So the most important consideration for me is whether we are >>>> locking >>>> ourselves into something that will be hard to deal with later or >>>> not. >>>> >>>> Should the jinja2 project fail by the wayside next year would we be >>>> able to easily replace it with another engine without changing the >>>> templates as stored ? >>>> >>>> Simo. >>>> >>> Ah, ok, I understand the concern. For now, the plan is that the >>> server >>> will do all the text processing, so I don't really forsee a need for >>> any >>> other client to read the mapping rules from LDAP. However, it's true >>> that templates written in jinja2 would probably need at least minor >>> changes to be compatible with another templating engine. (Same goes >>> for >>> any other choice - a lot of these engines seem to have very similar, >>> but >>> not exactly compatible, syntax). I don't really know how to judge >>> the >>> long-term viability of the jinja2 project, though it seems to be >>> recognized by lots of projects (ansible[1], openstack[2], flask[3], >>> even >>> django[4] which has its own templating engine). Other big users are exactly the reason why I consider Jinja2 to be a good choice. After all, if Jinja2 dies upstream we will not be alone :-) Petr^2 Spacek >>> In any case, if the team prefers it, I'd be comfortable going with a >>> more minimal DSL that only has the features we know we need. It >>> might >>> slightly limit the types of certs that can be generated, but that can >>> be >>> iterated on. But it would be another thing to design, build and >>> maintain. Let me know what you think. >> I am ok using jinja2 as long as we realize we may be on the hook for >> maintaining it ourselves in the long term. It's probably easier to do >> that than to write our own anyway. >> >> Simo. > It might also be worth pointing out that although the examples in the document > are based on jinja2, the overall approach should be pretty compatible with > other (existing or custom) template languages. So we're not locked into jinja2 > in that way, it's just the rules in deployed databases that would be tough to > convert. -- Petr^2 Spacek -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code