Re: explicit_bzero vs. alternatives
Philipp Klaus Krause wrote: > In the end, I wouldn't be surprised, if WG14 just goes with one of the > existing names, not caring about reserved identifiers. There is only one existing name in common use. Even glibc chose to go with the name explicit_bzero. I notice you keep using the other names, wilfully ignoring the reality on the ground. > But introducing a new name, from the reserved identifiers, for > functionality, for which implementation experience already exists using > a non-reserved name, still looks like a legitimate appoach to me. Legitimate like pulling a sword from a rock.
Re: explicit_bzero vs. alternatives
Theo de Raadt wrote in <61139.1597087...@cvs.openbsd.org>: |Philipp Klaus Krause wrote: |> Am 10.08.20 um 17:00 schrieb Theo de Raadt: |>> Philipp Klaus Krause wrote: |>> |>>> OpenBSD has the explicit_bzero function to reliably (i.e. even if not |>>> observable in the C abstract machine) overwrite memory with zeroes. |>>> |>>> WG14 is currently considering adding similar functionality to C2X. |>> |>> Then perhaps in the interests of the public they should use the same |>> name, but I suspect they won't. |> |> The functionality (i.e. some way to reliably overwrite memory) already |> exists under different names: explicit_bzero in OpenBSD | |This one was first. If i recall correctly others had already started using volatile pointers to memset(3) before. 'Kind of strange that bcopy etc. all were thrown away, just to bring back a bzero to circumvent overoptimization of compilers. That "reliably overwrite memory" .. bugs me, i know that ship has sailed, but if the programmer calls a function then for a reason. Sorry, the topic concerns me. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: explicit_bzero vs. alternatives
>> >> WG14 has reserved some identifiers for future extensions of the >> standard. E.g. those starting with mem_. Naturally, others then choose >> identifiers that do not conflict with this, such as explicit_bzero. But >> if that name is then used in the standard unchanged, it would mean that >> future extensions only use exactly those identifiers not reserved for >> future extensions. >> >> Philipp > > But if we would use reserved identifiers, we would be castigated for that. > > Don't you see your process does not work? > > -Otto > This is not the only aspect broken about reserved identifiers. C2X will introduce thousands of new functions (nearly all of them for floating-point), many with names not previously reserved. C++ has their namespaces, but there is no such solution for C. I don't know how to solve this problem, and I don't think there will be a solution anytime soon. In the end, I wouldn't be surprised, if WG14 just goes with one of the existing names, not caring about reserved identifiers. But introducing a new name, from the reserved identifiers, for functionality, for which implementation experience already exists using a non-reserved name, still looks like a legitimate appoach to me. Philipp
Re: explicit_bzero vs. alternatives
On Tue, Aug 11, 2020 at 08:20:32AM +0200, Otto Moerbeek wrote: > On Tue, Aug 11, 2020 at 08:13:24AM +0200, Philipp Klaus Krause wrote: > > > Am 11.08.20 um 02:52 schrieb Theo de Raadt: > > > > > > But no, WG14 are the lords and masters in the high castle, and now 6 > > > years after the ship sailed something Must Be Done, it must look like > > > They Solved The Problem, and so they'll create an incompatible API. > > > > > > Will they be heroes? No, not really. Changing the name is villainous. > > > > > > > The purpose of WG14 is to codify existing practise, not to invent (see > > N2086 http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2086.htm, 8. and > > 13.). > > > > WG14 has reserved some identifiers for future extensions of the > > standard. E.g. those starting with mem_. Naturally, others then choose > > identifiers that do not conflict with this, such as explicit_bzero. But > > if that name is then used in the standard unchanged, it would mean that > > future extensions only use exactly those identifiers not reserved for > > future extensions. > > > > Philipp > > But if we would use reserved identifiers, we would be castigated for that. > > Don't you see your process does not work? > > -Otto > Let me elaborate. IMO, if the WG refuses to include a common function user in various platforms into the standard and instead insist on naming it differently it just creates confusion and work and thus bugs. In that, it hinders progress instead of enabling it. -Otto
Re: explicit_bzero vs. alternatives
On Tue, Aug 11, 2020 at 08:13:24AM +0200, Philipp Klaus Krause wrote: > Am 11.08.20 um 02:52 schrieb Theo de Raadt: > > > > But no, WG14 are the lords and masters in the high castle, and now 6 > > years after the ship sailed something Must Be Done, it must look like > > They Solved The Problem, and so they'll create an incompatible API. > > > > Will they be heroes? No, not really. Changing the name is villainous. > > > > The purpose of WG14 is to codify existing practise, not to invent (see > N2086 http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2086.htm, 8. and > 13.). > > WG14 has reserved some identifiers for future extensions of the > standard. E.g. those starting with mem_. Naturally, others then choose > identifiers that do not conflict with this, such as explicit_bzero. But > if that name is then used in the standard unchanged, it would mean that > future extensions only use exactly those identifiers not reserved for > future extensions. > > Philipp But if we would use reserved identifiers, we would be castigated for that. Don't you see your process does not work? -Otto
Re: explicit_bzero vs. alternatives
Am 11.08.20 um 02:52 schrieb Theo de Raadt: > > But no, WG14 are the lords and masters in the high castle, and now 6 > years after the ship sailed something Must Be Done, it must look like > They Solved The Problem, and so they'll create an incompatible API. > > Will they be heroes? No, not really. Changing the name is villainous. > The purpose of WG14 is to codify existing practise, not to invent (see N2086 http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2086.htm, 8. and 13.). WG14 has reserved some identifiers for future extensions of the standard. E.g. those starting with mem_. Naturally, others then choose identifiers that do not conflict with this, such as explicit_bzero. But if that name is then used in the standard unchanged, it would mean that future extensions only use exactly those identifiers not reserved for future extensions. Philipp
Re: explicit_bzero vs. alternatives
Steffen Nurpmeso wrote: > Theo de Raadt wrote in > <61139.1597087...@cvs.openbsd.org>: > |Philipp Klaus Krause wrote: > |> Am 10.08.20 um 17:00 schrieb Theo de Raadt: > |>> Philipp Klaus Krause wrote: > |>> > |>>> OpenBSD has the explicit_bzero function to reliably (i.e. even if not > |>>> observable in the C abstract machine) overwrite memory with zeroes. > |>>> > |>>> WG14 is currently considering adding similar functionality to C2X. > |>> > |>> Then perhaps in the interests of the public they should use the same > |>> name, but I suspect they won't. > |> > |> The functionality (i.e. some way to reliably overwrite memory) already > |> exists under different names: explicit_bzero in OpenBSD > | > |This one was first. > > If i recall correctly others had already started using volatile > pointers to memset(3) before. Cannot take you seriously. A handful of instances where manual workaround were used cannot be honestly compared to an API used use 641 times in a complete operating system source tree. I was one of the people perpetuating those hacks. But the manual workarounds could not be sustainably "nitpicky" applied throughout a large source tree, and so we (openbsd and google developers) created a standard-ready API. That API has been adopted very substantially. But no, WG14 are the lords and masters in the high castle, and now 6 years after the ship sailed something Must Be Done, it must look like They Solved The Problem, and so they'll create an incompatible API. Will they be heroes? No, not really. Changing the name is villainous.
Re: explicit_bzero vs. alternatives
Tim van der Molen wrote: > Philipp Klaus Krause (2020-08-10 21:00 +0200): > > Am 10.08.20 um 17:00 schrieb Theo de Raadt: > > > Philipp Klaus Krause wrote: > > > > > >> OpenBSD has the explicit_bzero function to reliably (i.e. even if not > > >> observable in the C abstract machine) overwrite memory with zeroes. > > >> > > >> WG14 is currently considering adding similar functionality to C2X. > > > > > > Then perhaps in the interests of the public they should use the same > > > name, but I suspect they won't. > > > > The functionality (i.e. some way to reliably overwrite memory) already > > exists under different names: explicit_bzero in OpenBSD, > > explicit_bzero is also in glibc, musl, FreeBSD and DragonFly. And pervasively used. debian code search can review the situation. Will people need additional #ifdef's to use the new officially sanctioned API? To me, that seems utterly stupid.
Re: explicit_bzero vs. alternatives
Philipp Klaus Krause (2020-08-10 21:00 +0200): > Am 10.08.20 um 17:00 schrieb Theo de Raadt: > > Philipp Klaus Krause wrote: > > > >> OpenBSD has the explicit_bzero function to reliably (i.e. even if not > >> observable in the C abstract machine) overwrite memory with zeroes. > >> > >> WG14 is currently considering adding similar functionality to C2X. > > > > Then perhaps in the interests of the public they should use the same > > name, but I suspect they won't. > > The functionality (i.e. some way to reliably overwrite memory) already > exists under different names: explicit_bzero in OpenBSD, explicit_bzero is also in glibc, musl, FreeBSD and DragonFly. > memzero_explicit in Linux, I think that is in the Linux *kernel*. > memset_s in the optional Annex K of the C > standard, explicit_memset in NetBSD, SecureZeroMemory in Windows etc. > > A problem with the explicit_bzero name is that it is not an identifier > reserved for future extensions of the C standard, unlike identifiers > starting with mem. > > > > >> Considered options include: > >> > >> * A function like explicit_bzero or memset_explicit, that overwrites the > >> memory with a known value. > > > > We have never needed any value other than zero. > > Thanks. I assume this will help WG14. > > > >> * A function like memclear, that overwrites the memory in an > >> implementation-defined manner, possibly using random data. > > > > This option is pretty laughable, because the compiler has no way to > > collect random data. Their is nothing portable the compiler can call > > to get the random data. I've personally worked on making this possible > > for more than a decade, and it is still not all there. > > This option under the name secure_clear apparently is the one preferred > by WG21, the C++ comittee. > > Philipp
Re: explicit_bzero vs. alternatives
> Our base tree has 641 calls to explicit_bzero, because previous > commitees fucked the C language and created hundreds of not thousands of > security vulnerabilities by leaving data on the stack. In application > software, both explicit_memset and explicit Sorry I didn't finish the last sentence. In application software, both explicit_memset and memzero_explicit are used to some extent, usually security software or security-context libraries. Sometimes this occurs as direct calls, but more usually via a macro or function abstraction. But I am unable to quantify how often this happens.
Re: explicit_bzero vs. alternatives
Philipp Klaus Krause wrote: > Am 10.08.20 um 17:00 schrieb Theo de Raadt: > > Philipp Klaus Krause wrote: > > > >> OpenBSD has the explicit_bzero function to reliably (i.e. even if not > >> observable in the C abstract machine) overwrite memory with zeroes. > >> > >> WG14 is currently considering adding similar functionality to C2X. > > > > Then perhaps in the interests of the public they should use the same > > name, but I suspect they won't. > > The functionality (i.e. some way to reliably overwrite memory) already > exists under different names: explicit_bzero in OpenBSD This one was first. > memzero_explicit in Linux glibc people saw the need and followed us, but they felt compelled to change the name. NIH. > memset_s in the optional Annex K of the C > standard, Annex K is completely dead and un-adopted enough that application code will not even look for or call it. The embedding of errno_t and exceptions were some of the dumbest ideas ever, probably an attempt by MSFT to undermine unix-heritage libc functions. Because of those gross warts in the API, it will never see further adoption and noone should pay any attention to it. > explicit_memset in NetBSD Followed Linux, also a couple years later, and they needed to change the name also. > SecureZeroMemory in Windows etc. Which makes no attempt to be name it in a fashion that would ever gain cross-operating system use. Camel case functions do not make it into unix libraries. > A problem with the explicit_bzero name is that it is not an identifier > reserved for future extensions of the C standard, unlike identifiers > starting with mem. Well, the problem is we are not supposed to use the mem prefix, so WE DID NOT. It is like are suggesting it is OK that others simply stepped into the reserved name space? Maybe we should remember in the future, that if you don't stomp, the commitee will eventually arrive to create an #ifdef application horrorshow. Maybe the above mention of the ^mem prefixed one can be understood as a blessing, and we should have ignored the edit, and chosen such a name?? (We did ignore the edict when we created the widely deployed strlcpy/strlcat, and are probably doing the same with reallocarray/recallocarray/freezero which are now seeing substantial deployment also). > >> * A function like memclear, that overwrites the memory in an > >> implementation-defined manner, possibly using random data. > > > > This option is pretty laughable, because the compiler has no way to > > collect random data. Their is nothing portable the compiler can call > > to get the random data. I've personally worked on making this possible > > for more than a decade, and it is still not all there. > > This option under the name secure_clear apparently is the one preferred > by WG21, the C++ comittee. Well, I accuse the commitee of Not Invented Here. Our base tree has 641 calls to explicit_bzero, because previous commitees fucked the C language and created hundreds of not thousands of security vulnerabilities by leaving data on the stack. In application software, both explicit_memset and explicit It is obvious the commitee has no desire to focus on improving the ecosystem for their mistake and want to increase name confusion, but want to look good and solve this problem, what is it.. 10 or 15 years late? I've lost track of time. It is very much like the posix_random fiasco: https://www.austingroupbugs.net/view.php?id=859 So I predict it will take decades for secure_clear to come into common use. Sometimes a name isn't perfect. But one name may still be more perfect than two names, or three names, or six names. *SIX NAMES*, is the situation WG21 would create.
Re: explicit_bzero vs. alternatives
Am 10.08.20 um 17:00 schrieb Theo de Raadt: > Philipp Klaus Krause wrote: > >> OpenBSD has the explicit_bzero function to reliably (i.e. even if not >> observable in the C abstract machine) overwrite memory with zeroes. >> >> WG14 is currently considering adding similar functionality to C2X. > > Then perhaps in the interests of the public they should use the same > name, but I suspect they won't. The functionality (i.e. some way to reliably overwrite memory) already exists under different names: explicit_bzero in OpenBSD, memzero_explicit in Linux, memset_s in the optional Annex K of the C standard, explicit_memset in NetBSD, SecureZeroMemory in Windows etc. A problem with the explicit_bzero name is that it is not an identifier reserved for future extensions of the C standard, unlike identifiers starting with mem. > >> Considered options include: >> >> * A function like explicit_bzero or memset_explicit, that overwrites the >> memory with a known value. > > We have never needed any value other than zero. Thanks. I assume this will help WG14. > >> * A function like memclear, that overwrites the memory in an >> implementation-defined manner, possibly using random data. > > This option is pretty laughable, because the compiler has no way to > collect random data. Their is nothing portable the compiler can call > to get the random data. I've personally worked on making this possible > for more than a decade, and it is still not all there. This option under the name secure_clear apparently is the one preferred by WG21, the C++ comittee. Philipp
Re: explicit_bzero vs. alternatives
Philipp Klaus Krause wrote: > OpenBSD has the explicit_bzero function to reliably (i.e. even if not > observable in the C abstract machine) overwrite memory with zeroes. > > WG14 is currently considering adding similar functionality to C2X. Then perhaps in the interests of the public they should use the same name, but I suspect they won't. > Considered options include: > > * A function like explicit_bzero or memset_explicit, that overwrites the > memory with a known value. We have never needed any value other than zero. > * A function like memclear, that overwrites the memory in an > implementation-defined manner, possibly using random data. This option is pretty laughable, because the compiler has no way to collect random data. Their is nothing portable the compiler can call to get the random data. I've personally worked on making this possible for more than a decade, and it is still not all there. > Is there a rationale why OpenBSD went with their explicit_bzero design? > Were alternatives considered and rejected? Our rationale was coping with the C commitees changing C into an unsafe language. They decided to completely ignore risk factors associated with the memory state, in particular the exploitation of left-overs. In their high chairs, they decided concrete is strong enough and bridges don't need rebar. We built a function that cannot be skipped, and we chose a name for it. The two ideas you listed are overbuilt and silly.
explicit_bzero vs. alternatives
OpenBSD has the explicit_bzero function to reliably (i.e. even if not observable in the C abstract machine) overwrite memory with zeroes. WG14 is currently considering adding similar functionality to C2X. Considered options include: * A function like explicit_bzero or memset_explicit, that overwrites the memory with a known value. * A function like memclear, that overwrites the memory in an implementation-defined manner, possibly using random data. Is there a rationale why OpenBSD went with their explicit_bzero design? Were alternatives considered and rejected? Philipp
