Re: [gentoo-user] memset_s

[Jorge Almeida]

On Tue, Nov 14, 2017 at 8:42 PM, R0b0t1  wrote:
> On Tue, Nov 14, 2017 at 11:36 AM, Jorge Almeida  wrote:
>> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida  wrote:
>>
>>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>>
>>>
> Of course, what would really solve the optimize-into-oblivion problem
> is a pragma that when invoked on a particular block of code (maybe
> only a function definition) would tell the compiler to do what the
> programmer says rather than viewing a function as a kind of black box.
>

>>
>> It seems a solution exists with gcc:
>>
>> https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c
>>
>> The last reply:
>>
>> void __attribute__((optimize("O0"))) foo(unsigned char data) {
>> // unmodifiable compiler code
>> }
>>
>> Any comments, anyone? Yes, it's gcc, but IMO this should be in the
>> language itself. Am I right to assume this is a poorly known feature
>> of gcc?
>> It allows, for example,  to replace sensitive data by random bytes,
>> existing system callls like memset() or getrandom() can be used as
>> they are, no reimplementation needed.
>>
>
> Very interesting. I imagine the opinion of the standards committee
> would be that the variability in code generation precludes a standard
> interface to optimization controls. This might seem unusual, but
> languages with a very controlling standard (like Java or C#) are a new
> concept.

Well, we'll have to stick to gcc (or other compilers with the same
feature). OTOH, boldness doesn't seem to be the comittee's most
salient feature.

>
> What I am wondering about is if C code which uses
> __attribute__((optimize(...))) is against Gentoo package standards and
> would have to be removed from the Portage tree.
>


You can set your optimization preferences in make.conf, and still an
ebuild will override them if deemed unsafe. What would be the
difference?

Cheers

Jorge

Re: [gentoo-user] memset_s

[R0b0t1]

On Tue, Nov 14, 2017 at 11:36 AM, Jorge Almeida  wrote:
> On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida  wrote:
>
>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>
>>
 Of course, what would really solve the optimize-into-oblivion problem
 is a pragma that when invoked on a particular block of code (maybe
 only a function definition) would tell the compiler to do what the
 programmer says rather than viewing a function as a kind of black box.

>>>
>
> It seems a solution exists with gcc:
>
> https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c
>
> The last reply:
>
> void __attribute__((optimize("O0"))) foo(unsigned char data) {
> // unmodifiable compiler code
> }
>
> Any comments, anyone? Yes, it's gcc, but IMO this should be in the
> language itself. Am I right to assume this is a poorly known feature
> of gcc?
> It allows, for example,  to replace sensitive data by random bytes,
> existing system callls like memset() or getrandom() can be used as
> they are, no reimplementation needed.
>

Very interesting. I imagine the opinion of the standards committee
would be that the variability in code generation precludes a standard
interface to optimization controls. This might seem unusual, but
languages with a very controlling standard (like Java or C#) are a new
concept.

What I am wondering about is if C code which uses
__attribute__((optimize(...))) is against Gentoo package standards and
would have to be removed from the Portage tree.

Cheers,
 R0b0t1

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 12:09 PM, Jorge Almeida  wrote:

> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>
>
>>> Of course, what would really solve the optimize-into-oblivion problem
>>> is a pragma that when invoked on a particular block of code (maybe
>>> only a function definition) would tell the compiler to do what the
>>> programmer says rather than viewing a function as a kind of black box.
>>>
>>

It seems a solution exists with gcc:

https://stackoverflow.com/questions/2219829/how-to-prevent-gcc-optimizing-some-statements-in-c

The last reply:

void __attribute__((optimize("O0"))) foo(unsigned char data) {
// unmodifiable compiler code
}

Any comments, anyone? Yes, it's gcc, but IMO this should be in the
language itself. Am I right to assume this is a poorly known feature
of gcc?
It allows, for example,  to replace sensitive data by random bytes,
existing system callls like memset() or getrandom() can be used as
they are, no reimplementation needed.

Jorge Almeida

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Sun, Nov 12, 2017 at 7:03 PM, Mart Raudsepp  wrote:
> On L, 2017-11-11 at 00:10 +, Jorge Almeida wrote:
>> Well, most programmers probably won't care about this stuff anyway,
>> and people who deal with cryptography tend to be more cautious than
>> average. But I'm not really making a case for safe versions of known
>> functions. After all, the usual functions do fine for most
>> applications. memset() would be enough to clear RAM with sensitive
>> data if we had a pragma (or equivalent) to convince the compiler to
>> not ignore it (I mean a pragma to invoke on a particular function
>> definition when the programmer  feels that a black box behaviour is
>> undesirable). Of course, solving the problem of the compiler copying
>> stuff around might be harder nut to crack.
>
> Sounds like you want explicit_bzero from libbsd?
>
According to their man page, yes. I'll have to [try to] check the
source. I wonder how they do it? Even the volatile modifier doesn't
solve the problem, according to the link in previous post.

Anyway, there are probably circumstances where not optimizing a
particular function would be useful. Zeroing sensitive data is just
the first that comes to mind. (For example, what if you prefer
overwriting your sensitive data with random bytes?)

Jorge

Re: [gentoo-user] memset_s

[Mart Raudsepp]

On L, 2017-11-11 at 00:10 +, Jorge Almeida wrote:
> Well, most programmers probably won't care about this stuff anyway,
> and people who deal with cryptography tend to be more cautious than
> average. But I'm not really making a case for safe versions of known
> functions. After all, the usual functions do fine for most
> applications. memset() would be enough to clear RAM with sensitive
> data if we had a pragma (or equivalent) to convince the compiler to
> not ignore it (I mean a pragma to invoke on a particular function
> definition when the programmer  feels that a black box behaviour is
> undesirable). Of course, solving the problem of the compiler copying
> stuff around might be harder nut to crack.

Sounds like you want explicit_bzero from libbsd?

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 11:19 PM, R0b0t1  wrote:
> On Fri, Nov 10, 2017 at 2:09 PM, Jorge Almeida  wrote:
>> On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1  wrote:
>
>>
>> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>>
>
> I really think there is a deeper issue here then, which is that the
> compiler takes a lot of liberties when translating a program
> description into machine code. There have been suggestions made that
> this makes very nearly all compilers unsuitable for high reliability
> purposes. Cryptographic or user security code is likely a candidate
> for the label "high reliability."

Yes, the html page above has a link to a 2nd part, where (if I
understood correctly) it is concluded that currently there is no real
solution: even if the compiler does what it is told to, it may copy
data around, and the programmer has no control whatsoever over the
fate of such data.

>
> To further explain why the additions are counterproductive: the
> programmer still has to remember to use them. It is just as likely
> that the programmer will forget to use memset_s properly as any of the
> other functions in string.h (possibly by forgetting to sanitize input
> i.e. the memory segment boundaries).
>
Well, most programmers probably won't care about this stuff anyway,
and people who deal with cryptography tend to be more cautious than
average. But I'm not really making a case for safe versions of known
functions. After all, the usual functions do fine for most
applications. memset() would be enough to clear RAM with sensitive
data if we had a pragma (or equivalent) to convince the compiler to
not ignore it (I mean a pragma to invoke on a particular function
definition when the programmer  feels that a black box behaviour is
undesirable). Of course, solving the problem of the compiler copying
stuff around might be harder nut to crack.
>

>
> If you don't mind I might post this concern to the GCC mailing list,
> or you can take it up if you want.

Please do. I'm strictly amateur league, and you'll do a better job.
>
Cheers

Jorge

Re: [gentoo-user] memset_s

[R0b0t1]

On Fri, Nov 10, 2017 at 2:09 PM, Jorge Almeida  wrote:
> On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1  wrote:
>> Hello,
>>
>
>>
>> I'm having trouble finding the article again, but these functions look
>> very similar to Microsoft's extensions to the C standard. There is a
>> good case to be made that they are counterproductive.
>
> Yes, it looks like it. No wonder, if it's MS inspired. But what I care
> about is the fact that it's not optimized away, not the boundaries
> checking stuff. It's hard to believe that it is practically impossible
> to clean up a buffer, unless one is willing to forego all
> optimizations:
>
> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
>

I really think there is a deeper issue here then, which is that the
compiler takes a lot of liberties when translating a program
description into machine code. There have been suggestions made that
this makes very nearly all compilers unsuitable for high reliability
purposes. Cryptographic or user security code is likely a candidate
for the label "high reliability."

To further explain why the additions are counterproductive: the
programmer still has to remember to use them. It is just as likely
that the programmer will forget to use memset_s properly as any of the
other functions in string.h (possibly by forgetting to sanitize input
i.e. the memory segment boundaries).

>
>>> Of course, what would really solve the optimize-into-oblivion problem
>>> is a pragma that when invoked on a particular block of code (maybe
>>> only a function definition) would tell the compiler to do what the
>>> programmer says rather than viewing a function as a kind of black box.
>>>
>>
>> This would probably be useful. It may be wise to reimplement important
>> functionality.
>>
> No idea how difficult it would be to implement, of course. There might
> even exist a C keyword for that. After all, the C standard states the
> "as-if" rule, it might as well establish such an exception.
>

Sorry, I misrepresented what I meant. I meant to suggest
reimplementing, apart from a standard library, any critical code. This
is generally recommended against but unless there is a hand-tuned
version that has been guaranteed to work around quirks in your
compiler, you are now the person who has to write and maintain that
hand-tuned version.

If you don't mind I might post this concern to the GCC mailing list,
or you can take it up if you want.

Cheers,
 R0b0t1

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 4:25 PM, R0b0t1  wrote:
> Hello,
>

>
> I'm having trouble finding the article again, but these functions look
> very similar to Microsoft's extensions to the C standard. There is a
> good case to be made that they are counterproductive.

Yes, it looks like it. No wonder, if it's MS inspired. But what I care
about is the fact that it's not optimized away, not the boundaries
checking stuff. It's hard to believe that it is practically impossible
to clean up a buffer, unless one is willing to forego all
optimizations:

http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html


>> Of course, what would really solve the optimize-into-oblivion problem
>> is a pragma that when invoked on a particular block of code (maybe
>> only a function definition) would tell the compiler to do what the
>> programmer says rather than viewing a function as a kind of black box.
>>
>
> This would probably be useful. It may be wise to reimplement important
> functionality.
>
No idea how difficult it would be to implement, of course. There might
even exist a C keyword for that. After all, the C standard states the
"as-if" rule, it might as well establish such an exception.

Cheers

Jorge

Re: [gentoo-user] memset_s

[Alexander Kapshuk]

On Fri, Nov 10, 2017 at 6:25 PM, R0b0t1  wrote:

> Hello,
>
> On Fri, Nov 10, 2017 at 5:34 AM, Jorge Almeida 
> wrote:
> > On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet  wrote:
> >> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
> >>> I'm trying to use memset_s() but the system (glibc?) doesn't know
> >>> about it. I also tried to compile against musl, same result.
> >>>
> >
> >
> >> It seems as though it is simply not implemented, I found a variety of
> links
> >> that all support this:
> >>
> >> https://stackoverflow.com/a/40162721
> >>
> >> https://stackoverflow.com/questions/38322363/when-will-
> the-safe-string-functions-of-c11-be-part-of-glibc
> >>
> >> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
> >> implemented)
> >>
> >> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
> >>
> > OK, thanks. The last link even suggests that Annex K should be
> > deprecated. I suppose this people don't care about security at all.
> >
>
> I'm having trouble finding the article again, but these functions look
> very similar to Microsoft's extensions to the C standard. There is a
> good case to be made that they are counterproductive.
>
> > Of course, what would really solve the optimize-into-oblivion problem
> > is a pragma that when invoked on a particular block of code (maybe
> > only a function definition) would tell the compiler to do what the
> > programmer says rather than viewing a function as a kind of black box.
> >
>
> This would probably be useful. It may be wise to reimplement important
> functionality.
>
> Cheers,
>  R0b0t1
>
>
It's also a part of NetBSD's libc used by Apple.
https://opensource.apple.com/source/Libc/Libc-1044.1.2/string/NetBSD/memset_s.c

Re: [gentoo-user] memset_s

[R0b0t1]

Hello,

On Fri, Nov 10, 2017 at 5:34 AM, Jorge Almeida  wrote:
> On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet  wrote:
>> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
>>> I'm trying to use memset_s() but the system (glibc?) doesn't know
>>> about it. I also tried to compile against musl, same result.
>>>
>
>
>> It seems as though it is simply not implemented, I found a variety of links
>> that all support this:
>>
>> https://stackoverflow.com/a/40162721
>>
>> https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc
>>
>> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
>> implemented)
>>
>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
>>
> OK, thanks. The last link even suggests that Annex K should be
> deprecated. I suppose this people don't care about security at all.
>

I'm having trouble finding the article again, but these functions look
very similar to Microsoft's extensions to the C standard. There is a
good case to be made that they are counterproductive.

> Of course, what would really solve the optimize-into-oblivion problem
> is a pragma that when invoked on a particular block of code (maybe
> only a function definition) would tell the compiler to do what the
> programmer says rather than viewing a function as a kind of black box.
>

This would probably be useful. It may be wise to reimplement important
functionality.

Cheers,
 R0b0t1

Re: [gentoo-user] memset_s

[Jorge Almeida]

On Fri, Nov 10, 2017 at 10:52 AM, Marc Joliet  wrote:
> Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
>> I'm trying to use memset_s() but the system (glibc?) doesn't know
>> about it. I also tried to compile against musl, same result.
>>


> It seems as though it is simply not implemented, I found a variety of links
> that all support this:
>
> https://stackoverflow.com/a/40162721
>
> https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc
>
> https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not
> implemented)
>
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm
>
OK, thanks. The last link even suggests that Annex K should be
deprecated. I suppose this people don't care about security at all.

Of course, what would really solve the optimize-into-oblivion problem
is a pragma that when invoked on a particular block of code (maybe
only a function definition) would tell the compiler to do what the
programmer says rather than viewing a function as a kind of black box.

Jorge Almeida

Re: [gentoo-user] memset_s

[Marc Joliet]

Am Freitag, 10. November 2017, 10:54:53 CET schrieb Jorge Almeida:
> I'm trying to use memset_s() but the system (glibc?) doesn't know
> about it. I also tried to compile against musl, same result.
> 
> There's precious little info about memset_s in the net. Does it exist
> at all? No man page.
> 
> (https://www.cs.helsinki.fi/group/boi2016/doc/cppreference/reference/en.cppr
> eference.com/w/c/string/byte/memset.html)
> 
> What I tried:
> 
> #include 
> #include 
> #include 
> #define __STDC_WANT_LIB_EXT1__  1
> #include 
> #include 
> 
> int main(int argc, char** argv){
> #ifndef __STDC_LIB_EXT1__
> printf("CRAP\n");
> #else
> printf("COOL\n");
> #endif
> }
> 
> Compiled with -std=c11
> 
> You can guess what the output is.
> 
> Someone using it?
> 
> Jorge Almeida

It seems as though it is simply not implemented, I found a variety of links  
that all support this:

https://stackoverflow.com/a/40162721

https://stackoverflow.com/questions/38322363/when-will-the-safe-string-functions-of-c11-be-part-of-glibc

https://gcc.gnu.org/wiki/C11Status (which states that Annex K is not 
implemented)

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm

HTH
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup


signature.asc
Description: This is a digitally signed message part.

[gentoo-user] memset_s

[Jorge Almeida]

I'm trying to use memset_s() but the system (glibc?) doesn't know
about it. I also tried to compile against musl, same result.

There's precious little info about memset_s in the net. Does it exist
at all? No man page.

(https://www.cs.helsinki.fi/group/boi2016/doc/cppreference/reference/en.cppreference.com/w/c/string/byte/memset.html)

What I tried:

#include 
#include 
#include 
#define __STDC_WANT_LIB_EXT1__  1
#include 
#include 

int main(int argc, char** argv){
#ifndef __STDC_LIB_EXT1__
printf("CRAP\n");
#else
printf("COOL\n");
#endif
}

Compiled with -std=c11

You can guess what the output is.

Someone using it?

Jorge Almeida