Re: More on in-memory zeroisation

2007-12-18 Thread Peter Gutmann
Bodo Moeller <[EMAIL PROTECTED]> writes:
>On Sun, Dec 09, 2007 at 07:16:22PM +1300, Peter Gutmann wrote:
>> There was a discussion on this list a year or two back about problems in 
>> using
>> memset() to zeroise in-memory data, specifically the fact that optimising
>> compilers would remove a memset() on (apparently) dead data in the belief 
>> that
>> it wasn't serving any purpose.
>
>Actually this problem was discussed five years ago (October 2002) on the
>vuln-dev mailing list:

When I said "a year or two" I meant for large values of two.

Peter.

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-18 Thread Bodo Moeller
On Sun, Dec 09, 2007 at 07:16:22PM +1300, Peter Gutmann wrote:

> There was a discussion on this list a year or two back about problems in using
> memset() to zeroise in-memory data, specifically the fact that optimising
> compilers would remove a memset() on (apparently) dead data in the belief that
> it wasn't serving any purpose.

Actually this problem was discussed five years ago (October 2002) on
the vuln-dev mailing list:

http://www.securityfocus.com/archive/82/297827/30/0/threaded
http://msdn2.microsoft.com/en-us/library/ms972826.aspx

If the problem also was discussed here on the cryptography list a year
or two ago, I am afraid that my memory of this either has been zeroized,
or has been detected not to serve any purpose.

Bodo

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-14 Thread Werner Koch
On Thu, 13 Dec 2007 21:11, [EMAIL PROTECTED] said:

>   volatile char buf[SIZE];
>   /* ... do stuff with buf ... */
>   memset(buf, 0, sizeof(buf));

This has the little disadvantage that you need to check the attributes
of BUF first and that you can't immediately see what the memset is used
for.  For a long time we use the macros below to document the intention
and to make sure that the compiler does not do any harm:

  /* To avoid that a compiler optimizes certain memset calls away, these
 macros may be used instead. */
  #define wipememory2(_ptr,_set,_len) do { \
volatile char *_vptr=(volatile char *)(_ptr); \
size_t _vlen=(_len); \
while(_vlen) { *_vptr=(_set); _vptr++; _vlen--; } \
} while(0)
  #define wipememory(_ptr,_len) wipememory2(_ptr,0,_len)
  



Shalom-Salam,

   Werner


-- 
Die Gedanken sind frei.  Auschnahme regelt ein Bundeschgesetz.

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


RE: More on in-memory zeroisation

2007-12-14 Thread Leichter, Jerry
| I've been through the code.  As far as I can see, there's nothing in
| expand_builtin_memset_args that treats any value differently, so there
| can't be anything special about memset(x, 0, y).  Also as far as I can
| tell, gcc doesn't optimise out calls to memset, not even thoroughly
| dead ones
While good for existing crypto code, this is exactly the kind of thing
that's a problem.  We now have a well-distributed bit of folk knowledge
that memset(x,0,y) is treated specially by the compiler.  It isn't; this
"knowledge" is just repeated inaccurate rumor.  Fortunately, "not
treated specially" in this case defaults to a case that does what we
want - but it also means that if someone makes the "code has no effect"
analyzer smarter in some release of gcc, all of a sudden, these memset()
calls that we're relying on may suddenly just disappear from the
generated code.  How long before anyone notices?  It's not as if the
change log will show "optimize away dead calls to memset" - it will
likely contain some obscure comment like "improve recognition that type
B subtrees can be collapsed in phase 3".

The only *safe* way to write code like this - absent explicit support
in the standard - is with explicit support in each particular compiler.
Even something like:

#pragma always_call memset

ugly as it is, would work.

-- Jerry

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-14 Thread Thierry Moreau


Jack:

Thank you for pointing this. I must admit you point to an inescapable 
counter-example for my analysis.


Maybe global optimization was not a significant factor in the 1980's 
when the C standard language was established -- it does refer to 
external linkage and "genuine function".


In the case of volatile declaration, the GCC 4.2.2 compiler gave me a 
warning that the volatile qualifier was ignored because the memset 
formal parameter declaration does not match. At least, as a compiler 
user I get a proper warning message.


Regards

 - Thierry Moreau


Original message:

Jack Lloyd wrote:


On Wed, Dec 12, 2007 at 05:27:38PM -0500, Thierry Moreau wrote:

As a consequence of alleged consensus above, my understanding of the C 
standard would prevail and (memset)(?,0,?) would refer to an external 
linkage function, which would guarantee (to the sterngth of the above 
consensus) resetting an arbitrary memory area for secret intermediate 
result protection.



GCC on x86-64 (-O2) compiles this function to the same machine code
regardless of the value of ZEROIZE:

#include 

int sensitive(int key)
   {
   char buf[16];
   int result = 0;
   size_t j;

   for(j = 0; j != sizeof(buf); j++)
  buf[j] = key + j;

   for(j = 0; j != sizeof(buf); j++)
  result += buf[j];

#if ZEROIZE
   (memset)(buf, 0, sizeof(buf));
#endif

   return result;
   }

Even if (memset) must refer to a function with external linkage (an
analysis I find dubious), there is nothing stopping the compiler from
doing IPA/whole program optimization - especially with a very basic
function like memset (in the code above, if buf is declared volatile,
GCC does do the memset: but it does it by moving immediate zero values
directly to the memory locations, not by actually jumping to any
external function).

Regards,
  Jack


-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


RE: More on in-memory zeroisation

2007-12-14 Thread Dave Korn


  I've been through the code.  As far as I can see, there's nothing in
expand_builtin_memset_args that treats any value differently, so there can't be
anything special about memset(x, 0, y).  Also as far as I can tell, gcc doesn't
optimise out calls to memset, not even thoroughly dead ones: for example -


/artimi/software/firmware $ cat memstst.c

#include 
int foo (void);
int main (int argc, const char **argv)
{
  int var[100];
  memset (var, 0, sizeof var);
  foo ();
  return 0;
}

int foo (void)
{
  int var[100];
  memset (var, 0, sizeof var);
  return 0;
}

/artimi/software/firmware $ gcc -O2 memstst.c -o mt
/artimi/software/firmware $ gcc -O2 memstst.c -S -o memstst.s
/artimi/software/firmware $ grep memset memstst.s
call_memset
call_memset
.def_memset;.scl3;  .type   32; .endef
/artimi/software/firmware $


  This is not entirely unexpected; memset, even when expanded inline as a
builtin, still has libcall behaviour.

cheers,
  DaveK
-- 
Can't think of a witty .sigline today

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-14 Thread Alan Barrett
On Tue, 11 Dec 2007, Leichter, Jerry wrote:
> You can almost, but not quite, get the desired effect for memory zero-
> ization with "volatile".

I thought that this was guaranteed to work:

volatile char buf[SIZE];
/* ... do stuff with buf ... */
memset(buf, 0, sizeof(buf));

--apb (Alan Barrett)

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-14 Thread Jack Lloyd
On Wed, Dec 12, 2007 at 05:27:38PM -0500, Thierry Moreau wrote:
> As a consequence of alleged consensus above, my understanding of the C 
> standard would prevail and (memset)(?,0,?) would refer to an external 
> linkage function, which would guarantee (to the sterngth of the above 
> consensus) resetting an arbitrary memory area for secret intermediate 
> result protection.

GCC on x86-64 (-O2) compiles this function to the same machine code
regardless of the value of ZEROIZE:

#include 

int sensitive(int key)
   {
   char buf[16];
   int result = 0;
   size_t j;

   for(j = 0; j != sizeof(buf); j++)
  buf[j] = key + j;

   for(j = 0; j != sizeof(buf); j++)
  result += buf[j];

#if ZEROIZE
   (memset)(buf, 0, sizeof(buf));
#endif

   return result;
   }

Even if (memset) must refer to a function with external linkage (an
analysis I find dubious), there is nothing stopping the compiler from
doing IPA/whole program optimization - especially with a very basic
function like memset (in the code above, if buf is declared volatile,
GCC does do the memset: but it does it by moving immediate zero values
directly to the memory locations, not by actually jumping to any
external function).

Regards,
  Jack

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Thierry Moreau



Leichter, Jerry wrote:



On Wed, 12 Dec 2007, Thierry Moreau wrote:

| Date: Wed, 12 Dec 2007 16:24:43 -0500
| From: Thierry Moreau <[EMAIL PROTECTED]>
| To: "Leichter, Jerry" <[EMAIL PROTECTED]>
| Cc: Peter Gutmann <[EMAIL PROTECTED]>, cryptography@metzdowd.com
| Subject: Re: More on in-memory zeroisation
| 
| / testf.c /

| #include 
| #include 
| 
| typedef void *(*fpt_t)(void *, int, size_t);
| 
| void f(fpt_t arg)

| {
|   if (memset==arg)
|   printf("Hello world!\n");
| }
| 
| / test.c /

| #include 
| #include 
| 
| typedef void *(*fpt_t)(void *, int, size_t);
| 
| extern void f(fpt_t arg);
| 
| int main(int argc, char *argv[])

| {
|   f(memset);
|   return EXIT_SUCCESS;
| }
| 
| /*   I don't want to argue too theoretically.
| 
| - Thierry Moreau */

I'm not sure what you are trying to prove here.  Yes, I believe that
in most implementations, this will print "Hello world\n".  Is it,
however, a strictly conforming program (I think that's the right
standardese) - i.e., are the results guaranteed to be the same on
all conforming implementations?  I think you'll find it difficult
to prove that.


If there is a consensus among comforming implementation developers that 
the above program is comforming, that's a good enough "proof" for me.


As a consequence of alleged consensus above, my understanding of the C 
standard would prevail and (memset)(?,0,?) would refer to an external 
linkage function, which would guarantee (to the sterngth of the above 
consensus) resetting an arbitrary memory area for secret intermediate 
result protection.


Reading ANSI X3.159-1989, I believe there would be such a consensus, and 
I find it quite obvious. You may disagree, and I will no further argument.


Regards,

--

- Thierry Moreau

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Leichter, Jerry


On Wed, 12 Dec 2007, Thierry Moreau wrote:

| Date: Wed, 12 Dec 2007 16:24:43 -0500
| From: Thierry Moreau <[EMAIL PROTECTED]>
| To: "Leichter, Jerry" <[EMAIL PROTECTED]>
| Cc: Peter Gutmann <[EMAIL PROTECTED]>, cryptography@metzdowd.com
| Subject: Re: More on in-memory zeroisation
| 
| / testf.c /
| #include 
| #include 
| 
| typedef void *(*fpt_t)(void *, int, size_t);
| 
| void f(fpt_t arg)
| {
|   if (memset==arg)
|   printf("Hello world!\n");
| }
| 
| / test.c /
| #include 
| #include 
| 
| typedef void *(*fpt_t)(void *, int, size_t);
| 
| extern void f(fpt_t arg);
| 
| int main(int argc, char *argv[])
| {
|   f(memset);
|   return EXIT_SUCCESS;
| }
| 
| /*   I don't want to argue too theoretically.
| 
| - Thierry Moreau */
I'm not sure what you are trying to prove here.  Yes, I believe that
in most implementations, this will print "Hello world\n".  Is it,
however, a strictly conforming program (I think that's the right
standardese) - i.e., are the results guaranteed to be the same on
all conforming implementations?  I think you'll find it difficult
to prove that.

BTW, it *might* not even be true in practice if you build your program
as multiple shared libraries!
-- Jerry



-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Thierry Moreau

/ testf.c /
#include 
#include 

typedef void *(*fpt_t)(void *, int, size_t);

void f(fpt_t arg)
{
if (memset==arg)
printf("Hello world!\n");
}

/ test.c /
#include 
#include 

typedef void *(*fpt_t)(void *, int, size_t);

extern void f(fpt_t arg);

int main(int argc, char *argv[])
{
f(memset);
return EXIT_SUCCESS;
}

/*   I don't want to argue too theoretically.

- Thierry Moreau */

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Leichter, Jerry
| > If the function is defined as I suggested - as a static or inline -
| > you can, indeed, takes its address.  (In the case of an inline, this
| > forces the compiler to materialize a copy somewhere that it might
| > not otherwise have produced, but not to actually *use* that copy,
| > except when you take the address.)  You are allowed to invoke the
| > function using the address you just took.  However, what in that
| > tells you that the compiler - knowing exactly what code will be
| > invoked - can't elide the call?
| 
| Case of static function definition: the standard says that standard
| library headers *declare* functions, not *define* them.
Where does it say it *can't* define them?  How could a Standard-conforming
program tell the difference?  If no Standard-conforming program can
tell the difference between two implementations, it makes no difference
what you, as an omniscient external observer, might know - they are either
both compatible with the Standard, or neither is.

| Case of inline: I don't know if inline definition falls in the
| standard definition of declaration.
It makes not difference.

| Also, the standard refers to these identifiers as external
| linkage. This language *might* not creare a mandatory provision if
| there was a compelling reason to have static or inline implementation,
| but I doubt the very infrequent use of (memset)(?,0,?) instead of
| memset(?,0,?) is a significant optimization opportunity. The compiler
| writer risks a non-compliance assessment in making such strectched
| reading of the standard in the present instance, for no gain in any
| benchmark or production software speed measurement.
| 
| Obviously, a pointer to an external linkage scope function must adhere
| to the definition of pointer equality (==) operator.
What do you think "the definition of pointer equality" actually is?  Keep
in mind that you need to find the definition *in the Standard*.  The
*mathematical* definition is irrelevant.

| Maybe a purposedly stretched reading of the standard might let you
| make your point. I don't want to argue too theoretically. Peter and I
| just want to clear memory!
Look, I write practical programs all the time - mainly in C++ recently,
but the same principles apply.  My programs tend to be broadly portable
across different compilers and OS's.  I've been doing this for close to
30 years.  I stick to the published standards where possible, but there's
no way to avoid making assumptions that go beyond the standards in a few
cases:  Every standard I know of is incomplete, and no implementation I've
ever worked with is *really* 100% compliant.

It's one thing to point out a set of practical techniques for getting
certain kinds of things done.  It's another to make unsupportable arguments
that those practical techniques are guaranteed to work.  There's tons of
threaded code out there, for example.  Given the lack of any discussion of
threading in existing language standards, most of them skate on thin ice.
Some things are broadly agreed upon, and "quality of implementation"
requirements make it unlikely that a compiler will break them.  Other
things are widely believed by developers to have been agreed upon, but
have *not* really be agreed upon by providers.  Programs that rely on
these things - e.g., that C++ function-scope static initializers will be
run in a thread-safe way - will fail here and there, because in fact
compiler developers don't even try to support them.  Because of the ever-
growing importance of threaded programs, this situation is untenable, and
in fact the language groups are starting to grapple with how to incorporate
threads.

Security issues are a similar issue.  The fact is, secure programming
sometimes requires primitives that the standards simply don't provide.
Classic example:  For a long time, there was *no* safe way to use
sprintf(), since there was no a priori way of determining how long the
output string might be.  People had various hacks, but all of them could
be fooled, unless you pretty much re-implemented sprintf() yourself.
snprintf() fixed that.

There is, today, no way to guarantee that memset() will be run, within the
confines of the standard.  This is a relatively minor oversight - C has
seen such issues as important since volatile was introduced well before
the language was standardized.  I expect we'll see some help on this in
a future version.  In the meanwhile, it would be nice if compiler
developers would agree on some extra-Standard mechanisms.  The gcc hack
could be a first step - but it should be written down, not just something
a few insiders know about.  Standards are supposed to grow by standardi-
zing proven practice, not by innovation.

The problem with unsupportable assumptions that some hack or another
provides a solution is that they block *actual* solutions.  By all means
use them where necessary - but push for better approaches.

-- Jerry

| Kind reg

Re: More on in-memory zeroisation

2007-12-13 Thread Thierry Moreau



Leichter, Jerry wrote:


If the function is defined as I suggested - as a static or inline - you
can, indeed, takes its address.  (In the case of an inline, this forces
the compiler to materialize a copy somewhere that it might not otherwise
have produced, but not to actually *use* that copy, except when you take
the address.)  You are allowed to invoke the function using the address
you just took.  However, what in that tells you that the compiler -
knowing exactly what code will be invoked - can't elide the call?


Case of static function definition: the standard says that standard 
library headers *declare* functions, not *define* them.


Case of inline: I don't know if inline definition falls in the standard 
definition of declaration.


Also, the standard refers to these identifiers as external linkage. This 
language *might* not creare a mandatory provision if there was a 
compelling reason to have static or inline implementation, but I doubt 
the very infrequent use of (memset)(?,0,?) instead of memset(?,0,?) is a 
significant optimization opportunity. The compiler writer risks a 
non-compliance assessment in making such strectched reading of the 
standard in the present instance, for no gain in any benchmark or 
production software speed measurement.


Obviously, a pointer to an external linkage scope function must adhere 
to the definition of pointer equality (==) operator.


Maybe a purposedly stretched reading of the standard might let you make 
your point. I don't want to argue too theoretically. Peter and I just 
want to clear memory!


Kind regards,


--

- Thierry Moreau

CONNOTECH Experts-conseils inc.
9130 Place de Montgolfier
Montreal, Qc
Canada   H2M 2A1

Tel.: (514)385-5691
Fax:  (514)385-5900

web site: http://www.connotech.com
e-mail: [EMAIL PROTECTED]

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Leichter, Jerry
| > However, that doesn't say anything about whether f is actually
| > invoked at run time.  That comes under the "acts as if" rule:  If
| > the compiler can prove that the state of the C (notional) virtual
| > machine is the same whether f is actually invoked or not, it can
| > elide the call.  Nothing says that memset() can't actually be
| > defined in the appropriate header, as a static (or, in C99, inline)
| > function.
| 
| The standard actually says "... it is permitted to take the address of
| a library function even if it is defined as a macro ...". The standard
| works for me as a source code author who needs an execution-aware
| memcpy function from time to time. Overworked GCC contributors should
| work to comply to the standard, not to address Peter, Thierry, and
| whoever's wildests dreams.
If the function is defined as I suggested - as a static or inline - you
can, indeed, takes its address.  (In the case of an inline, this forces
the compiler to materialize a copy somewhere that it might not otherwise
have produced, but not to actually *use* that copy, except when you take
the address.)  You are allowed to invoke the function using the address
you just took.  However, what in that tells you that the compiler -
knowing exactly what code will be invoked - can't elide the call?

By the way, you might wonder what happens if two different CU's take
the address of memset and we then compare them.  In this kind of
implementation, they will be unequal - but in fact nothing in the
Standard says they can't be!  A clever compiler could have all kinds
of reasons to produce multiple copies of the same function.  All you
can say is that if two function pointers are equal, they point to the
same function.  No converse form is provable within the Standard.

You might try something like:

typedef (void *(*memset_ptr)(const void*,int,size_t));

volatile memset_ptr p_memset = &(memset);

(I *think* I got that syntax right!)

Then you can invoke (*p_memset).  But if you do this in the same
compilation unit, a smart compiler that does value propagation could
determine that it knows where p_memset points, and that it knows what
the code there is, so it can go ahead and do its deeper analysis.

Using:

volatile memset_ptr p_memset = &(memset);

in one compilation unit and then:

extern volatile memset_ptr p_memset = &(memset);

will keep you safe from single-CU optimizations, but nothing in the
Standard says that's all there are.  Linker-based optimizations
could have the additional information that nowhere in the program
can p_memset be changed, and further that p_memset is allocated to
regular memory, and in principle the calls could be elided at that
point.  Mind you, I would be astounded if any compiler/linker system
actually attempted such an optimization ... but that doesn't make
it illegal within the language of the Standard.

| > Then the compiler can look at the implementation and "prove"
| > that a memset() to a dead variable can be elided
| 
| It can't prove much in the case of (memset)()
In principle (I'll grant you, probably not in practice), it can
provie quite a bit - and certainly enough to justify eliding the
call.
-- Jerry

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


RE: More on in-memory zeroisation

2007-12-13 Thread Leichter, Jerry
| > Then the compiler can look at the implementation and "prove" that a
| > memset() to a dead variable can be elided
| 
| One alternative is to create zero-ing functions that wrap memset()
| calls with extra instructions that examine some of the memory, log a
| message and exit the application if the memory is not zero. This has
| two benefits: 1) It guarantees the compiler will leave the memset() in
| place and 2) guarantees the memset() worked. It does incur a few extra
| instructions though.
| 
| I guess it is possible that the compiler would somehow optimize the
| memset to only zero the elements subsequent code compares... Hmmm
| [Of course your application could be swapped out and just before the
| memset call writing your valuable secrets to the system swap file on
| disk... :-( ]
In practice, with an existing compiler you are not in a position to
change, these kinds of games are necessary.  If you're careful, you look
at the generated code to make sure it does what you expect.

But this is a very bad - and potentially very dangerous - approach.
You're relying on the stupidity of the compiler - and on the compiler
not become more intelligent over time.  Are you really prepared to
re-check the generated code every time the compiler is rev'ed?

There sometimes needs to be an explicit way to tell the compiler that
some operation *must* be done in some way, no matter what the compiler
thinks it knows.  There's ample precedent for this.  For example,
floating point arithmetic doesn't exactly follow the usual laws of
arithmetic (e.g., it's not associative, if you consider overflows), some
if you know what you are doing in construction an FP algorithm, you have
to have a way to tell the compiler "Yes, I know you think you can
improve my code here, but just leave it alone, thank you very much."
And all programming languages that see numerical programming as within
their rubric provide standardized, documented ways to do just that.  C
have "volatile" so that you can tell the compiler that it may not elide
or move operations on a variable, even when those operations have no
effects visible in the C virtual machine.  (The qualifier was added
to support memory-mapped I/O, where there can be locations that look
like memory but have arbitrarily different semantics from normal
memory.)  And so on.

You can almost, but not quite, get the desired effect for memory zero-
ization with "volatile".  Something more is needed, and software that
will be used to write cryptographic algorithms needs access to that
"something more" (to be pinned down explicitly).

-- Jerry

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-13 Thread Thierry Moreau



Leichter, Jerry wrote:


| > There was a discussion on this list a year or two back about
| > problems in using memset() to zeroise in-memory data, specifically
| > the fact that optimising compilers would remove a memset() on
| > (apparently) dead data in the belief that it wasn't serving any
| > purpose.
| 
| Then, s/memset(?,0,?)/(memset)(?,0,?)/ to get rid of compiler

| in-lining.
| 
| Ref: ANSI X3.159-1989, section 4.1.6 (Use of C standard library

| functions)
I don't have te C89 spec handy (just the C99 spec, which is laid
out differently), but from what I recall, this construct guarantees
nothing of the sort.

Most standard library functions can be implemented as macros.  Using the
construct (f)(args) guarantees that you get the actual function f,
rather than the macro f.


Indeed, the actual function, memcpy in the present instance.

memcpy, the actual function, is aware of the execution environment, 
because it is part of the run-time library. The compiler is not as 
deeply aware of the execution environment.


The source code construct (f)(args) is provided by the standard to allow 
the program to explicitly rely on the actual library function.


At least this is my understanding of compiler optimization techniques as 
subordinate to standard definition of the C language. I too often turned 
off optimization due to (suspected) optimized-in crash, I would like to 
rely on the (f)(args) to locally turn off optimization.



However, that doesn't say anything about
whether f is actually invoked at run time.  That comes under the "acts
as if" rule:  If the compiler can prove that the state of the C
(notional) virtual machine is the same whether f is actually invoked or
not, it can elide the call.  Nothing says that memset() can't actually
be defined in the appropriate header, as a static (or, in C99, inline)
function.


The standard actually says "... it is permitted to take the address of a 
library function even if it is defined as a macro ...". The standard 
works for me as a source code author who needs an execution-aware memcpy 
function from time to time. Overworked GCC contributors should work to 
comply to the standard, not to address Peter, Thierry, and whoever's 
wildests dreams.



Then the compiler can look at the implementation and "prove"
that a memset() to a dead variable can be elided


It can't prove much in the case of (memset)()


Regards,

--

- Thierry Moreau

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-11 Thread Leichter, Jerry
| > There was a discussion on this list a year or two back about
| > problems in using memset() to zeroise in-memory data, specifically
| > the fact that optimising compilers would remove a memset() on
| > (apparently) dead data in the belief that it wasn't serving any
| > purpose.
| 
| Then, s/memset(?,0,?)/(memset)(?,0,?)/ to get rid of compiler
| in-lining.
| 
| Ref: ANSI X3.159-1989, section 4.1.6 (Use of C standard library
| functions)
I don't have te C89 spec handy (just the C99 spec, which is laid
out differently), but from what I recall, this construct guarantees
nothing of the sort.

Most standard library functions can be implemented as macros.  Using the
construct (f)(args) guarantees that you get the actual function f,
rather than the macro f.  However, that doesn't say anything about
whether f is actually invoked at run time.  That comes under the "acts
as if" rule:  If the compiler can prove that the state of the C
(notional) virtual machine is the same whether f is actually invoked or
not, it can elide the call.  Nothing says that memset() can't actually
be defined in the appropriate header, as a static (or, in C99, inline)
function.  Then the compiler can look at the implementation and "prove"
that a memset() to a dead variable can be elided

-- Jerry

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


RE: More on in-memory zeroisation

2007-12-10 Thread Dave Korn
On 09 December 2007 06:16, Peter Gutmann wrote:

> Reading through "Secure Programming with Static Analysis", I noticed an
> observation in the text that newer versions of gcc such as 3.4.4 and 4.1.2
> treat the pattern:
> 
>   "memset(?, 0, ?)"
> 
> differently from any other memset in that it's not optimised out.

> Can anyone who knows more about gcc development provide more insight on
> this? Could it be made an official, supported feature of the compiler?

  I'm sure it could; why not raise it on the GCC mailing list?  It sounds like
all it would involve would be a patch to the documentation and maybe a comment
in the source.

cheers,
  DaveK
-- 
Can't think of a witty .sigline today

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-10 Thread Florian Weimer
* Thierry Moreau:

> Peter Gutmann wrote:
>
>> There was a discussion on this list a year or two back about problems in 
>> using
>> memset() to zeroise in-memory data, specifically the fact that optimising
>> compilers would remove a memset() on (apparently) dead data in the belief 
>> that
>> it wasn't serving any purpose.
>>
>
> Then, s/memset(?,0,?)/(memset)(?,0,?)/ to get rid of compiler in-lining.
>
> Ref: ANSI X3.159-1989, section 4.1.6 (Use of C standard library functions)

This isn't true; inlining of standard library functions is always
permitted under the as-if rule.

-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]


Re: More on in-memory zeroisation

2007-12-10 Thread Thierry Moreau



Peter Gutmann wrote:


There was a discussion on this list a year or two back about problems in using
memset() to zeroise in-memory data, specifically the fact that optimising
compilers would remove a memset() on (apparently) dead data in the belief that
it wasn't serving any purpose.



Then, s/memset(?,0,?)/(memset)(?,0,?)/ to get rid of compiler in-lining.

Ref: ANSI X3.159-1989, section 4.1.6 (Use of C standard library functions)


--

- Thierry Moreau


-
The Cryptography Mailing List
Unsubscribe by sending "unsubscribe cryptography" to [EMAIL PROTECTED]