Re: Reordering new API's that have a libctx, propq
There are many red herrings in here, and I would argue that trying to be too uniform in the way you think about all functions may be harmful, because not all functions are classified the same. We cannot deny that many of our interfaces have an OOP flair. We may be programming in C, but we very obviously have that kind of pattern, a "poor man's OOP" in C. So speaking about our interfaces in OOP terms is not far fetched, as long as we keep in mind that we can only take the argument so far. I would argue that EVP_XXX_get_whatever() / EVP_XXX_get_whatever() are classic accessors, and I assume that we have all been brought up with the description of "this" as the hidden first argument to any class method in C++, so it's not very far fetched to emulate that in C by making the instance you want details from or change details of as the (explicit) first argument. I would further argue that EVP_XXX_fetch() is a constructor (of an instance of EVP_XXX), and that with the assumption that there is no "this" in a constructor, the discussion about the arguments and their order must be different. What I hear, even though not in such terms, is an argument of the what the library context is and how that should affect the interface. In relation to EVP_XXX_fetch(), it seems to be more of a factory (or strictly speaking, the pool of resources, with hidden factory functions), and depending on the factory model you lean on, it might or might not be sensible to have it as first argument. My you, I'm trying quite hard to see it from a fresh user experience (as far as I can imagine it... after all, 20ish years with my fingers deep in OpenSSL entrails makes you not so fresh). I think, BTW, that this really comes down to how we view the library context. So far, I've seen all these interpretations (not all said explicitly, but clearly visible in code or how we argue about it): - framework - scope (I'm unsure if that differs much from framework) - factory / factory pool - bag of resources Personally, I have zero problems viewing it as all of these combined, but that requires us to be clear on how it's used in different places. Cheers, Richard On Sat, 05 Sep 2020 23:18:21 +0200, Tim Hudson wrote: > > > On Sun, Sep 6, 2020 at 6:55 AM Richard Levitte wrote: > > I'd rank the algorithm name as the most important, it really can't do > anything of value without it. > > It also cannot do anything without knowing which libctx to use. Look at the > implementation. > Without the libctx there is no "from-where" to specify. > > This is again hitting the concept of where do things come from and what is a > default. > Once "global" disappears as such, logically everything comes from a libctx. > > Your argument is basically "what" is more important than "from" or "where". > And the specific context here is where you see "from" or "where" can be > defaulted to a value so it > can be deduced so it isn't (as) important in the API. > > That would basically indicate you would (applying the same pattern/rule in a > different context) > change: > > int EVP_PKEY_get_int_param(EVP_PKEY *pkey, const char *key_name, int *out); > > To the following (putting what you want as the most important rather than > from): > > int EVP_PKEY_get_int_param(char *key_name, EVP_PKEY *pkey, int *out); > > Or pushing it right to the end after the output parameter: > > int EVP_PKEY_get_int_param(char *key_name, int *out,EVP_PKEY *pkey); > > The context of where things come from is actually the most critical item in > any of the APIs we > have. > Even though what you want to get from where you want to get it is in the > point of the API call you > need to specify where from first as that context sets the frame of the call. > > Think of it around this way - we could have an implementation where we > remember the last key that > we have used and allow you to simply indicate you use the last key or if we > didn't want the last > key used to be able to specify it via a non-NULL pointer. This isn't that > unusual an API but not > something I'm suggesting we add - just trying to get the point across that > you are still thinking > of global and libctx as something super special with an exception in its > handling rather than > applying a general rule which is pretty much what we use everywhere else. > > And in which case where you generally don't provide a reference as there is > some default meaning > for it in general and can provide a reference for that sort of API would this > make sense to you: > > int EVP_PKEY_get_int_param(char *key_name, int *out,EVP_PKEY *pkey); > > If pkey is NULL then you use the last key that you referenced, if it is not > then you use the > specified pkey. For the application the specific key_name is the most > important thing (using your > argument that basically states the "what" is what counts). > > I would suggest that you really would still want to place the EVP_PKEY first > - even if you had a
Re: Reordering new API's that have a libctx, propq
On Sun, Sep 6, 2020 at 6:55 AM Richard Levitte wrote: > I'd rank the algorithm name as the most important, it really can't do > anything of value without it. > It also cannot do anything without knowing which libctx to use. Look at the implementation. Without the libctx there is no "from-where" to specify. This is again hitting the concept of where do things come from and what is a default. Once "global" disappears as such, logically everything comes from a libctx. Your argument is basically "what" is more important than "from" or "where". And the specific context here is where you see "from" or "where" can be defaulted to a value so it can be deduced so it isn't (as) important in the API. That would basically indicate you would (applying the same pattern/rule in a different context) change: *int EVP_PKEY_get_int_param(EVP_PKEY *pkey, const char *key_name, int *out);* To the following (putting what you want as the most important rather than from): *int EVP_PKEY_get_int_param(char *key_name, EVP_PKEY *pkey, int *out);* Or pushing it right to the end after the output parameter: *int EVP_PKEY_get_int_param(char *key_name, int *out,EVP_PKEY *pkey);* The context of where things come from is actually the most critical item in any of the APIs we have. Even though what you want to get from where you want to get it is in the point of the API call you need to specify where from first as that context sets the frame of the call. Think of it around this way - we could have an implementation where we remember the last key that we have used and allow you to simply indicate you use the last key or if we didn't want the last key used to be able to specify it via a non-NULL pointer. This isn't that unusual an API but not something I'm suggesting we add - just trying to get the point across that you are still thinking of global and libctx as something super special with an exception in its handling rather than applying a general rule which is pretty much what we use everywhere else. And in which case where you generally don't provide a reference as there is some default meaning for it in general and can provide a reference for that sort of API would this make sense to you: *int EVP_PKEY_get_int_param(char *key_name, int *out,EVP_PKEY *pkey);* If pkey is NULL then you use the last key that you referenced, if it is not then you use the specified pkey. For the application the specific key_name is the most important thing (using your argument that basically states the "what" is what counts). I would suggest that you really would still want to place the EVP_PKEY first - even if you had a defaulting mechanism of referring to the last key used. Conceptually you always have to have knowledge of from-where when you are performing a function. And that *context* is what is the most important. Tim.
Re: Reordering new API's that have a libctx, propq
Hi, so... "importance" quite obviously carries different meaning to different people. What I see below is the meaning "having the longest life span" or possibly "being the biggest and most powerful resource". I've a different interpretation of "importance". Looking at EVP_XXX_fetch(), it's primary function is to get the caller an implementation of an algorithm, so a pretty important input for it to function is the name of that algorithm. Of course, it also needs to know where to go look and under what conditions (i.e. what properties need to apply), but in terms of importance for this function to work, I'd rank the algorithm name as the most important, it really can't do anything of value without it. With the in mind, the current (libctx, algoname, propq) argument order is... odd. I don't quite see if there was a suggestion to have (libctx, propq, algoname) as argument order; that's just plain weird in my mind. Those were late evening thoughts, I'll get back when I'm mulled over this a bit more. Cheers, Richard On Sat, 05 Sep 2020 12:44:51 +0200, Nicola Tuveri wrote: > > > On Sat, Sep 5, 2020, 12:13 Tim Hudson wrote: > > On Sat, Sep 5, 2020 at 6:38 PM Nicola Tuveri wrote: > > In most (if not all) cases in our functions, both libctx and > propquery are optional > arguments, as we have global defaults for them based on the loaded > config file. > As such, explicitly passing non-NULL libctx and propquery, is likely > to be an exceptional > occurrence rather than the norm. > > And that is where we have a conceptual difference, the libctx is always > used. If it is > provided as a NULL parameter then it is a shortcut to make the call to > get the default or to > get the already set one. > Conceptually it is always required for the function to operate. > > And the conceptual issue is actually important here - all of these > functions require the > libctx to do their work - if it is not available then they are unable to > do their work. > We just happen to have a default-if-NULL. > > If C offered the ability to default a parameter if not provided (and many > languages offer > that) I would expect we would be using it. > But it doesn't - we are coding in C. > > So it is really where-is-what-this-function-needs-coming-from that > actually is the important > thing - the source of the information the function needs to make its > choice. > It isn't which algorithm is being selected - the critical thing is from > which pool of > algorithm implementations are we operating. The pool must be specified > (this is C code), but > we have a default value. > > And that is why I think the conceptual approach here is getting confused > by the arguments > appearing to be optional - conceptually they are not - we just have a > defaulting mechanism and > that isn't the same conceptually as the arguments actually being optional. > > Clearer? > > It's not yet clear to me the distinction you are trying to make. > > I'll try to spell out what I extrapolated from your answer, and I apologize > in advance if I am > misunderstanding your argument, please be patient and correct me in that case > so I can better > understand your point! > > It seems to me you are making a conceptual difference between > - a function that internally requires an instance of foo to work (and has a > default if foo is > given as NULL among the arguments); e.g libctx is necessary for a fetch, if a > NULL libctx is given > a mechanism is in place to retrieve the global default one > - a function that internally uses an instance of foo only if a non-NULL one > is passed as argument; > e.g. bnctx, if the user provides it this is used by the callee and passed to > its callee, if the > user passes NULL the function creates a fresh one for itself and/or its > callees > > But as a consumer of the API that difference is not visible and probably not > even interesting, as > we are programming in C and passing pointers, there are certain arguments > that are required and > must be passed as valid pointers, others that appear optional because as a > consumer of that API I > can pass NULL and let the function internally default to a reasonable > behavior (and whatever this > "reasonable behavior" is — whether the first or the second case from above, > or another one I did > not list —, it's part of the documentation of that API). > > IMHO, in the consumer POV, libctx and propq are optional arguments (even in C > where optional or > default arguments do not technically exist and the caller needs to always > specify a value for each > argument, which are always positional) in the sense that they can pass NULL > as a value rather than > a pointer to a fully instantiated object of the required type. > Even more so given that, excluding a minority of cases, we can expect > consumers of the AP
Re: Reordering new API's that have a libctx, propq
On Sat, Sep 5, 2020, 14:01 Tim Hudson wrote: > On Sat, Sep 5, 2020 at 8:45 PM Nicola Tuveri wrote: > >> Or is your point that we are writing in C, all the arguments are >> positional, none is ever really optional, there is no difference between >> passing a `(void*) NULL` or a valid `(TYPE*) ptr` as the value of a `TYPE >> *` argument, so "importance" is the only remaining sorting criteria, hence >> (libctx, propq) are always the most important and should go to the >> beginning of the args list (with the exception of the `self/this` kind of >> argument that always goes first)? >> > > That's a reasonable way to express things. > > The actual underlying point I am making is that we should have a rule in > place that is documented and that works within the programming language we > are using and that over time doesn't turn into a mess. > We do add parameters (in new function names) and we do like to keep the > order of the old function - and ending up with a pile of things in the > "middle" is in my view is one of the messes that we should be avoiding. > We are already adding new functions, with the ex suffix, to allow users to keep using the old version, so given that the users passing to the "_ex" function are already altering their source, why are we limiting us from rationalizing the signature from the PoV of new users and old users alike that are in general quite critic of our API usability, even if it means that applying both "required-ness" and "importance" as sorting criteria sometime we end up adding in the middle? I don't think I am being dismissive of the needs of existing applications here: if a maintainer is altering their code from using "EVP_foo()" to "EVP_foo_ex()", they will likely also be looking at the documentation of the old and the new API versions and there shouldn't be really any additional significanf cost in adding a parameter a the edges or in the middle. I think the importance argument is the one that helps for setting order and > on your "required args" coming first approach the importance argument also > applies because it is actually a required argument simply with an implied > default - which again puts it not at the end of the argument list. The > library context is required - always - there is just a default - and that > parameter must be present because we are working in C. > I think I disagree with this, from the API CONSUMER PoV there is a clear difference between a function where they don't need to pass a valid libctx pointer and instead pass NULL (because there is a default associated with passing NULL) and a function like an hypothetical OSSL_LIBCTX_get_this(libctx) or OSSL_LIBCTX_set_that(libctx, value). In the first case the function operates despite the programmer not providing an explicit libctx (because a default one is used), in the latter the programmer is querying/altering directly the libctx and one must be provided. *… actually only now that I wrote out the greyed text above I think I am starting to understand what you meant all along!* Your point is that any API that uses a `libctx` (and `propq`) is always querying/altering a libctx, even if we use NULL as a shortcut to mean "the global one", so if we are fetching an algorithm, getting/setting something on a libctx scope, creating a new object, we are always doing so from a given libctx. In that sense, when an API consumer is passing NULL they are not passing "nothing" (on which my "optional arguments" approach is based), but they are passing NULL as a valid reference to a very specific libctx. I now see what you meant, and I can see how it is a relevant thing to make evident also in the function signature/usage. But I guess we can also agree that passing NULL as a very specific reference instead of as "nothing", can create a lot of confusion for external consumers of the API, if it took this long for one of us — *ok, it's me, so probably everyone else understood your point immediately, but still…* — to understand the difference you were pointing out, even knowing how these functions work internally and being somewhat familiar with dealing with libctx-s. If we want to convey this difference properly to our users, would it be better to use a new define? #define OSSL_DEFAULT ((void*)42) - OSSL_DFLT could be a shorter alternative, if we prefer brevity. - I am intentionally not specifying _LIBCTX as we might reuse a similar mechanism to avoid overloading NULL in other similar situations: e.g. for propq, where NULL is again a shortcut for the global ones, compared with passing "" as an empty properties query, seems like another good candidate for using a symbol to explicitly highlight that the default propq will be used - I would avoid making OSSL_DFLT an alias for NULL to prevent users from sidestepping the define completely and use NULL directly (because "it just works anyway"): that is why I am picking 42 as an example, but any magic number different from NULL/0 and that is most likely/always tre
Re: Reordering new API's that have a libctx, propq
On Sat, Sep 5, 2020 at 8:45 PM Nicola Tuveri wrote: > Or is your point that we are writing in C, all the arguments are > positional, none is ever really optional, there is no difference between > passing a `(void*) NULL` or a valid `(TYPE*) ptr` as the value of a `TYPE > *` argument, so "importance" is the only remaining sorting criteria, hence > (libctx, propq) are always the most important and should go to the > beginning of the args list (with the exception of the `self/this` kind of > argument that always goes first)? > That's a reasonable way to express things. The actual underlying point I am making is that we should have a rule in place that is documented and that works within the programming language we are using and that over time doesn't turn into a mess. We do add parameters (in new function names) and we do like to keep the order of the old function - and ending up with a pile of things in the "middle" is in my view is one of the messes that we should be avoiding. I think the importance argument is the one that helps for setting order and on your "required args" coming first approach the importance argument also applies because it is actually a required argument simply with an implied default - which again puts it not at the end of the argument list. The library context is required - always - there is just a default - and that parameter must be present because we are working in C. Whatever it is that we end up decided to do, the rule should be captured and should also allow for the fact that we will evolve APIs and create _ex versions and those two will also evolve and a general rule should be one that doesn't result in an inconsistent treatment for argument order as we add _ex versions. Tim.
Re: Reordering new API's that have a libctx, propq
On Sat, Sep 5, 2020, 12:13 Tim Hudson wrote: > On Sat, Sep 5, 2020 at 6:38 PM Nicola Tuveri wrote: > >> In most (if not all) cases in our functions, both libctx and propquery >> are optional arguments, as we have global defaults for them based on the >> loaded config file. >> As such, explicitly passing non-NULL libctx and propquery, is likely to >> be an exceptional occurrence rather than the norm. >> > > And that is where we have a conceptual difference, the libctx is *always* > used. > If it is provided as a NULL parameter then it is a shortcut to make the > call to get the default or to get the already set one. > Conceptually it is always required for the function to operate. > > And the conceptual issue is actually important here - all of these > functions require the libctx to do their work - if it is not available then > they are unable to do their work. > We just happen to have a default-if-NULL. > > If C offered the ability to default a parameter if not provided (and many > languages offer that) I would expect we would be using it. > But it doesn't - we are coding in C. > > So it is really where-is-what-this-function-needs-coming-from that > actually is the important thing - the source of the information the > function needs to make its choice. > It isn't which algorithm is being selected - the critical thing is from > which pool of algorithm implementations are we operating. The pool must be > specified (this is C code), but we have a default value. > > And that is why I think the conceptual approach here is getting confused > by the arguments appearing to be optional - conceptually they are not - we > just have a defaulting mechanism and that isn't the same conceptually as > the arguments actually being optional. > > Clearer? > It's not yet clear to me the distinction you are trying to make. I'll try to spell out what I extrapolated from your answer, and I apologize in advance if I am misunderstanding your argument, please be patient and correct me in that case so I can better understand your point! It seems to me you are making a conceptual difference between - a function that internally requires an instance of foo to work (and has a default if foo is given as NULL among the arguments); e.g libctx is necessary for a fetch, if a NULL libctx is given a mechanism is in place to retrieve the global default one - a function that internally uses an instance of foo only if a non-NULL one is passed as argument; e.g. bnctx, if the user provides it this is used by the callee and passed to its callee, if the user passes NULL the function creates a fresh one for itself and/or its callees But as a consumer of the API that difference is not visible and probably not even interesting, as we are programming in C and passing pointers, there are certain arguments that are required and must be passed as valid pointers, others that appear optional because as a consumer of that API I can pass NULL and let the function internally default to a reasonable behavior (and whatever this "reasonable behavior" is — whether the first or the second case from above, or another one I did not list —, it's part of the documentation of that API). IMHO, in the consumer POV, libctx and propq are optional arguments (even in C where optional or default arguments do not technically exist and the caller needs to always specify a value for each argument, which are always positional) in the sense that they can pass NULL as a value rather than a pointer to a fully instantiated object of the required type. Even more so given that, excluding a minority of cases, we can expect consumers of the APIs taking libctx and propq as arguments to pass NULL for both of them and rely on the openssl config mechanism. So while I agree with Tim that sometime it is valuable to make a difference among the consequences of passing NULL as arguments in the context of one kind of function and another, I believe the place for that is the documentation not its signature. The signature of the function should be designed for consumer usability, and the conventional pattern there seems to be - required args - optional args - callback+cb_args and inside each group the "importance" factor should be the secondary sorting criteria. "importance" is probably going to be affected by the difference you are making (or my understanding of it): e.g. if a function took both libctx and bnctx, the fact that a valid pre-existing libctx is required to work (and a global already existing one will be retrieved in case none is given), while a fresh short-lived bnctx is going to be created only for the lifetime of the called function in case none is given seems to indicate that libctx is of vital importance for the API functionality, while bnctx is of minor relevance. But... going this way as a generalized approach, would bring us to the "add in the middle" scenario that we'd like to avoid. I recognize that this is a point you already made in your original writeup, as the te
OpenSSL is looking for a full time Administrator and Manager
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 The OpenSSL Management Committee are looking to hire a full time Administrator and Manager. Details of the role can be found here: https://www.openssl.org/blog/blog/2020/09/05/OpenSSL.ProjectAdminRole/ To apply please send your cover letter and resume to j...@openssl.org by 20th September 2020. Regards, The OpenSSL Project Team -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEhlersmDwVrHlGQg52cTSbQ5gRJEFAl9TVxgACgkQ2cTSbQ5g RJHKGggAn1YGhR7UwtgVXTMWUKiv4jYpXd5OaHonAaUwIFdkXUzBmmEq9PP1Thw/ A4rQ/anDZ6SfRlFaGxQB1Fyz5LRyNDhHA48lM0v/Yw55S6NfSrMaPcGRuU8Odikf 4Nd7zzD3RcOgfhphdHEXz7ykMi90ATVcLTVnaoQtkvw5LHeiXzqzBLT9+WEcENWU 4z2WLJRGTpwIBfYfm6/NQPTDzsy/VBoVW/nl1mx6jkvL2UxuOdp4rfTMz9lu3IPk CnkujXxDIVSn02xSiRccj3ujnFqOq4lwtSiOzOl/HowlCDY6DmRhIvsu1PnPzJ15 v5JbQhDpk4kHjalCsJq2QfdcP41pDQ== =h9gU -END PGP SIGNATURE-
Re: Reordering new API's that have a libctx, propq
On Sat, Sep 5, 2020 at 6:38 PM Nicola Tuveri wrote: > In most (if not all) cases in our functions, both libctx and propquery are > optional arguments, as we have global defaults for them based on the loaded > config file. > As such, explicitly passing non-NULL libctx and propquery, is likely to be > an exceptional occurrence rather than the norm. > And that is where we have a conceptual difference, the libctx is *always* used. If it is provided as a NULL parameter then it is a shortcut to make the call to get the default or to get the already set one. Conceptually it is always required for the function to operate. And the conceptual issue is actually important here - all of these functions require the libctx to do their work - if it is not available then they are unable to do their work. We just happen to have a default-if-NULL. If C offered the ability to default a parameter if not provided (and many languages offer that) I would expect we would be using it. But it doesn't - we are coding in C. So it is really where-is-what-this-function-needs-coming-from that actually is the important thing - the source of the information the function needs to make its choice. It isn't which algorithm is being selected - the critical thing is from which pool of algorithm implementations are we operating. The pool must be specified (this is C code), but we have a default value. And that is why I think the conceptual approach here is getting confused by the arguments appearing to be optional - conceptually they are not - we just have a defaulting mechanism and that isn't the same conceptually as the arguments actually being optional. Clearer? Tim.
Re: Reordering new API's that have a libctx, propq
Thanks Tim for the writeup! I tend to agree with Tim's conclusions in general, but I fear the analysis here is missing an important premise that could influence the outcome of our decision. In most (if not all) cases in our functions, both libctx and propquery are optional arguments, as we have global defaults for them based on the loaded config file. As such, explicitly passing non-NULL libctx and propquery, is likely to be an exceptional occurrence rather than the norm. For optional parameters most developers from C and a variety of languages, would expect them to come at the end of the list of parameters, and this also follows the rule of thumb of "importance" used by Tim to pick 1) and B/A). For this reason I would instead suggest to go with 2) and C) in this case (with the caveat of keeping callback and its args as the very last arguments, again this is a non-written convention not only for us but quite widespread). Nicola On Sat, Sep 5, 2020, 10:10 Tim Hudson wrote: > I place the OTC hold because I don't believe we should be making parameter > reordering decisions without actually having documented what approach we > want to take so there is clear guidance. > This was the position I expressed in the last face to face OTC meeting - > that we need to write such things down - so that we avoid this precise > situation - where we have new functions that are added that introduce the > inconsistency that has been noted here that PR#12778 is attempting to > address. > > However, this is a general issue and not a specific one to OPENSSL_CTX and > it should be discussed in the broader context and not just be a last minute > (before beta1) API argument reordering. > That does not provide anyone with sufficient time to consider whether or > not the renaming makes sense in the broader context. > I also think that things like API argument reordering should have been > discussed on openssl-project so that the broader OpenSSL community has an > opportunity to express their views. > > Below is a quick write up on APIs in an attempt to make it easier to hold > an email discussion about the alternatives and implications of the various > alternatives over time. > I've tried to outline the different options. > > In general, the OpenSSL API approach is of the following form: > > > > *rettype TYPE_get_something(TYPE *,[args])rettype TYPE_do_something(TYPE > *,[args])TYPE*TYPE_new([args])* > > This isn't universally true, but it is the case for the majority of > OpenSSL functions. > > In general, the majority of the APIs place the "important" parameters > first, and the ancillary information afterwards. > > In general, for output parameters we tend to have those as the return > value of the function or an output parameter that > tends to be at the end of the argument list. This is less consistent in > the OpenSSL APIs. > > We also have functions which operate on "global" information where the > information used or updated or changed > is not explicitly provided as an API parameter - e.g. all the byname > functions. > > Adding the OPENSSL_CTX is basically providing where to get items from that > used to be "global". > When performing a lookup, the query string is a parameter to modify the > lookup being performed. > > OPENSSL_CTX is a little different, as we are adding in effectively an > explicit parameter where there was an implicit (global) > usage in place. But the concept of adding parameters to functions over > time is one that we should have a policy for IMHO. > > For many APIs we basically need the ability to add the OPENSSL_CTX that is > used to the constructor so that > it can be used for handling what used to be "global" information where > such things need the ability to > work with other-than-the-default OPENSSL_CTX (i.e. not the previous single > "global" usage). > > That usage works without a query string - as it isn't a lookup as such - > so there is no modifier present. > For that form of API usage we have three choices as to where we place > things: > > 1) place the context first > > *TYPE *TYPE_new(OPENSSL_CTX *,[args])* > > 2) place the context last > > *TYPE *TYPE_new([args],OPENSSL_CTX *)* > > 3) place the context neither first nor last > > *TYPE *TYPE_new([some-args],OPENSSL_CTX *,[more-args])* > > Option 3 really isn't a sensible choice to make IMHO. > > When we are performing effectively a lookup that needs a query string, we > have a range of options. > If we basically state that for a given type, you must use the OPENSSL_CTX > you have been provided with on construction, > (not an unreasonable position to take), then you don't need to modify the > existing APIs. > > If we want to allow for a different OPENSSL_CTX for a specific existing > function, then we have to add items. > Again we have a range of choices: > > A) place the new arguments first > > > *rettype TYPE_get_something(OPENSSL_CTX *,TYPE *,[args])rettype > TYPE_do_something(OPENSSL_CTX *,TYPE *,[args])* >
Re: Reordering new API's that have a libctx, propq
Thanks for the writeup.. > My view is that the importance of arguments is what sets their order - and > that is why for the TYPE functions the TYPE pointer > comes first. We could have just as easily specified it as last or some other > order, but we didn’t. Isn’t that the order that the fetch is in if it changes to what it is in the PR? I would argue that the algorithm is the most important parameter in the fetch. Deciding what argument is more ‘important’ is not always easy either. If we are doing an operation such as a get or load and it needs a whole lot of input params in order to return something, then aren’t they more important/ or just as important as the libctx,propq - (since the libctx,propq is just used normally to fetch something)? This then means the libctx ends up last, or maybe in the middle again. Also it is quite common for the new() methods (such as for X509) to need both the libctx and propq.. The X509 object gets passed to an d2i method that internally needs to do fetches (SHA1) and cache. Shane > On 5 Sep 2020, at 5:09 pm, Tim Hudson wrote: > > My view is that the importance of arguments is what sets their order - and > that is why for the TYPE functions the TYPE pointer > comes first. We could have just as easily specified it as last or some other > order, but we didn't.
Re: Reordering new API's that have a libctx, propq
I place the OTC hold because I don't believe we should be making parameter reordering decisions without actually having documented what approach we want to take so there is clear guidance. This was the position I expressed in the last face to face OTC meeting - that we need to write such things down - so that we avoid this precise situation - where we have new functions that are added that introduce the inconsistency that has been noted here that PR#12778 is attempting to address. However, this is a general issue and not a specific one to OPENSSL_CTX and it should be discussed in the broader context and not just be a last minute (before beta1) API argument reordering. That does not provide anyone with sufficient time to consider whether or not the renaming makes sense in the broader context. I also think that things like API argument reordering should have been discussed on openssl-project so that the broader OpenSSL community has an opportunity to express their views. Below is a quick write up on APIs in an attempt to make it easier to hold an email discussion about the alternatives and implications of the various alternatives over time. I've tried to outline the different options. In general, the OpenSSL API approach is of the following form: *rettype TYPE_get_something(TYPE *,[args])rettype TYPE_do_something(TYPE *,[args])TYPE*TYPE_new([args])* This isn't universally true, but it is the case for the majority of OpenSSL functions. In general, the majority of the APIs place the "important" parameters first, and the ancillary information afterwards. In general, for output parameters we tend to have those as the return value of the function or an output parameter that tends to be at the end of the argument list. This is less consistent in the OpenSSL APIs. We also have functions which operate on "global" information where the information used or updated or changed is not explicitly provided as an API parameter - e.g. all the byname functions. Adding the OPENSSL_CTX is basically providing where to get items from that used to be "global". When performing a lookup, the query string is a parameter to modify the lookup being performed. OPENSSL_CTX is a little different, as we are adding in effectively an explicit parameter where there was an implicit (global) usage in place. But the concept of adding parameters to functions over time is one that we should have a policy for IMHO. For many APIs we basically need the ability to add the OPENSSL_CTX that is used to the constructor so that it can be used for handling what used to be "global" information where such things need the ability to work with other-than-the-default OPENSSL_CTX (i.e. not the previous single "global" usage). That usage works without a query string - as it isn't a lookup as such - so there is no modifier present. For that form of API usage we have three choices as to where we place things: 1) place the context first *TYPE *TYPE_new(OPENSSL_CTX *,[args])* 2) place the context last *TYPE *TYPE_new([args],OPENSSL_CTX *)* 3) place the context neither first nor last *TYPE *TYPE_new([some-args],OPENSSL_CTX *,[more-args])* Option 3 really isn't a sensible choice to make IMHO. When we are performing effectively a lookup that needs a query string, we have a range of options. If we basically state that for a given type, you must use the OPENSSL_CTX you have been provided with on construction, (not an unreasonable position to take), then you don't need to modify the existing APIs. If we want to allow for a different OPENSSL_CTX for a specific existing function, then we have to add items. Again we have a range of choices: A) place the new arguments first *rettype TYPE_get_something(OPENSSL_CTX *,TYPE *,[args])rettype TYPE_do_something(OPENSSL_CTX *,TYPE *,[args])* B) place the new arguments first after the TPYE *rettype TYPE_get_something(TYPE *,OPENSSL_CTX *,[args])rettype TYPE_do_something(TYPE *,OPENSSL_CTX *,[args])* C) place the new arguments last *rettype TYPE_get_something(TYPE *,[args], OPENSSL_CTX *)rettype TYPE_do_something(TYPE *,[args], OPENSSL_CTX *)* D) place the new arguments neither first nor last *rettype TYPE_get_something(TYPE *,[some-args], OPENSSL_CTX *,[more-args])rettype TYPE_do_something(OPENSSL_CTX *,TYPE *,[some-args], OPENSSL_CTX *,[more-args])* Option D really isn't a sensible choice to make IMHO. My view is that the importance of arguments is what sets their order - and that is why for the TYPE functions the TYPE pointer comes first. We could have just as easily specified it as last or some other order, but we didn't. Now when we need to add a different location from which to retrieve other information we need to determine where this gets added. I'd argue that it is at constructor time that we should be adding any OPENSSL_CTX or query parameter for any existing TYPE usage in OpenSSL. If we feel the need to cross OPENSSL_CTX's (logically that is what is being done) in
