I talked a bit with Daniel about this off-list. Here's a summary of where we got to:
1. In the current "new-order" flow, the server doesn't *really* have to be proactive -- instead of updating an order object when an authz changes state, it can wait to update/issue until the client polls for the order object, indicating that it's actually interested in having a certificate. That doesn't alleviate the pain of storing the CSR, but it at least avoids the expense of finding affected orders and issuing in cases where the client abandons the order. If we think this approach is OK, I would propose that we revise the document to make clear that it is OK. What do folks think? Daniel noted that there might be some issues with GET idempotency here, but I don't think this actually makes GET non-idempotent. 2. Some CAs might want to support only the "new-authz" flow, i.e., never allow a "new-order" request if the relevant authorizations haven't already been done. It seems like this could be addressed in a pretty straightforward manner just by defining a new error code (say "preauthRequired") that the server can return in this case. Personally, it seems to me that if we make the clarification (1) and add the error code (2) described here, that would address the pain points Daniel raises, without a whole lot of new mechanism. I'm not really seeing much additional value to be had from an explicit "ok issue now" request (vs. the GET poll). --Richard On Wed, Sep 20, 2017 at 1:22 PM, Richard Barnes <[email protected]> wrote: > Hey Daniel, > > Thanks for the input. I can see why you might be feeling some pain here, > but I'm not sure the proposed solution is quite right. > > Note that ACME already has the "new-authorization" flow, which does pretty > much exactly what you're proposing. The only difference is that instead of > authorizations being created all at once in response to a "new-cert" > request, they're created individually by the client. But it's ultimately > the same number of RTTs (authz fetch ~ new-authz) -- or rather, one fewer > in the new-authz case because there's no new-cert request. To what degree > would your problem be solved simply by having the affected integrators move > over to that flow? > > I can sympathize that the new-order flow might not be suitable for all use > cases. Maybe it would be useful to write down some thoughts on which use > cases are best addressed by which flows. > > --Richard > > > > On Tue, Sep 19, 2017 at 12:26 PM, Daniel McCarney <[email protected]> > wrote: > >> Hi folks, >> >> Just over a year ago in a thread titled "Proactive vs On-Finalization >> Certificate Issuance"[0] we reached the consensus on the question of >> whether to >> issue a certificate "on-finalization" or "proactively" with the >> conclusion to >> standardize on proactive issuance. >> >> Implementation experience shows that proactive issuance is slow in >> practice. An >> order can be associated with many authorizations and an authorization can >> be >> associated with many orders. This interacts poorly with optimizations >> Let's >> Encrypt has developed from in-the-field experience with ACME and it is >> difficult >> to rate-limit effectively. I propose that we return to the >> on-finalization style >> approach and that we provide the CSR during finalization.. >> >> ## Issues with proactive issuance >> >> ### Authorization Reuse >> >> Proactive issuance requires that when an authorization is switched to a >> valid >> state by the result of a challenge update that we find the associated >> orders and >> iterate each order's authorizations to see if all of the authorizations >> are >> valid and issuance can occur. >> >> Let's Encrypt reuses existing authorizations[1][2] whenever possible, as >> encouraged by ACME. This has been of huge value in reducing resource >> consumption >> (both in terms of database storage and challenge/API requests). Many >> clients >> create pending authorizations they do not finalize and similarly re-create >> pending authorizations for identifiers that the account already has valid >> authorizations for. Unchecked this leads to significant database growth >> and >> resource consumption, even with aggressive rate-limits[3]. >> >> The combination of authorization reuse and proactive issuance means a >> given >> authorization can be associated with many orders. This necessitates an >> expensive >> many-to-many query at each authorization update to see if any associated >> orders are now complete. >> >> One possible workaround is a rate-limit that would constrain the number of >> orders an authorization can be associated with. However establishing the >> value >> of the limit to both reduce server-side load & also effectively support >> Let's >> Encrypt's large-volume integrators and their issuance patterns is >> challenging. >> >> ### CSR-first new-order flow >> >> To support proactive issuance the CA also needs to have the CSR for the >> order >> at-hand when checking each authorization in case issuance can proceed. The >> `new-order` request that creates the order therefore carries the full CSR >> that >> was previously delivered by the "certificate finalize" message. >> >> We see users create many more authorizations than they are able to >> finalize >> successfully. With the legacy "new-cert" flow our authorization reuse >> dramatically helped reduce the DB space used up by authorizations that >> will >> never lead to issuance because no CSR is stored until all identifiers are >> validated. >> >> With the new-order flow we have to accept and store a full CSR before >> handing >> back authorizations to the client. Reuse of authorizations does not >> prevent the >> DB bloat that will occur from users submitting new order requests and not >> fulfilling them. The CSR contains a full public key and is one of the >> single >> largest contributors to DB disk space usage. Addressing this will require >> another strict rate limit on outstanding new-orders and is again >> difficult to >> “dial-in” in to both prevent excessive resource consumption and also >> maintain >> the ability for big integrators to process large-scale issuance using our >> API. >> >> # Proposed Change >> >> I believe we should change the `new-order` endpoint to not accept a full >> CSR but >> to instead accept a list of identifiers that the ACME client wishes to >> authenticate & issue for. Authorizations can be created and returned for >> those >> domains as happens today. >> >> The order resource should be changed to have a “finalizeURL” field which >> provides the client a URL to POST a CSR to indicate that the CA should >> issue the >> certificate. Proactive issuance should be removed outright. >> >> This will address both the problem of scanning existing many-to-many >> orders >> & their authorizations as well as the db bloat that will occur from >> front-loading the delivery of the CSR. >> >> It may be tempting to leave proactive issuance as an optional feature but >> this >> will require maintaining the CSR in the new-order and I also believe that >> having >> two very distinct methods of issuance will complicate the client >> ecosystem and >> returns us to the question of how to determine which will occur that >> spawned the >> original "Proactive vs On-finalization Certificate Issuance" >> thread[0]. >> >> [0] https://mailarchive.ietf.org/arch/msg/acme/0lVmGl8e-rmSH0x7y >> cDW5dj6GAY >> [1] https://community.letsencrypt.org/t/upcoming-api-changes/17947 >> [2] https://community.letsencrypt.org/t/automatic-recycling-of-p >> ending-authorizations/41321 >> [3] https://letsencrypt.org/docs/rate-limits/ >> >> >> _______________________________________________ >> Acme mailing list >> [email protected] >> https://www.ietf.org/mailman/listinfo/acme >> >> >
_______________________________________________ Acme mailing list [email protected] https://www.ietf.org/mailman/listinfo/acme
