Re: GnuTLS (libgrypt really) and Postfix
On Thu, Feb 16, 2006 at 09:36:16AM +1000, James A. Donald wrote: In the case in question, going bad means that the program appears to be encrypting data, but is NOT encrypting data, or is only trivially encrypting data. This is far worse for the customer than an encryption program that simply aborts. Who said the program's primary function is data security? The program may be handling entirely public data, available from multiple sources. One of the sources may offer opportunistic encrypt, which though potentially useful, is inessential and mostly does no harm. It takes hubris for libgrypt to *assume* that is functions are essential. The dichotomy between pretending to encrypt data and exiting is false. It is plainly correct to not encrypt any data and say so (return errors from the encryption API). I'm outta here. I did not expect any controversy on this point, and don't expect views to shift dramatically. If the developers were open to the issue, the request might have been fruitful. If they dig in their heels, I am free to use other libraries. -- /\ ASCII RIBBON NOTICE: If received in error, \ / CAMPAIGN Victor Duchovni please destroy and notify X AGAINST IT Security, sender. Sender does not waive / \ HTML MAILMorgan Stanley confidentiality or privilege, and use is prohibited. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Tue, Feb 14, 2006 at 04:26:35PM -0500, Steven M. Bellovin wrote: In message [EMAIL PROTECTED], Werner Koch writes: I agree. However the case at hand is a bit different. I can't imagine how any application or upper layer will be able to recover from that error (ENOENT when opening /dev/random). Okay, the special file might just be missing and a mknod would fix that ;-). Is it the duty of an application to fix an incomplete installation - how long shall this be taken - this is not the Unix philosophy. It can take context-specific error recovery. Maybe that's greying out the encrypt button on a large GUI. Maybe it's paging the system administrator. It can run 'mknod' inside the appropriate chroot partition, much as /sbin/init on some systems creates /dev/console. It can symlink /dev/geigercounter to /dev/random. It can load the kernel module that implements /dev/random. It can do a lot of things that may be more appropriate than exiting. Or an even simpler example: maybe it will still be a fatal error, but there's some important state outside the library being called that it should clean up before exiting so abruptly. Somehow, applications that are consumers of crypto libraries seem like likely candidates for this sort of thing. -- Dan. pgpJyhp2aeO8S.pgp Description: PGP signature
Re: GnuTLS (libgrypt really) and Postfix
On Tue, Feb 14, 2006 at 01:00:33PM -0500, Steven M. Bellovin wrote: We all agree that critical errors like this should be caught; the only question is at what layer the action should take place. I'm an adherent to the Unix philosophy -- when a decision is made at a lower level, it takes away the ability of the higher level to do something different if appropriate, and this loss of flexibility is a bad thing. Thanks, this makes the point very clearly! Let me suggest a C-compatible possibility: pass an extra parameter to the library routines, specifying a procedure to call if serious errors occur. If that pointer is null, the library can abort. The pass-a-function pointer approach covers the simpler cases. Large utility libraries (OpenSSL, Kerberos, ...) sometimes have a tougher problem to solve. - The function needs error detail arguments so it can take the right actions. - Errors may need a classification system, so that new errors of the same type can be handled generically in legacy code as the library evolves. - The function needs an application context argument so it has access to the data it needs to take the right actions. So, the more sophisticated C-language designs (e.g. OpenSSL or Kerberos) include an error management API. These are clearly work-arounds for lack of real exceptions. They take care to design and implement, and it may be difficult or impractical to retrofit an existing design that did not pay the price from the start, but I find claims that the exit() approach is best *on architectural grounds* rather surprising... ERR_get_error(3)OpenSSL ERR_get_error(3) NAME ERR_get_error, ERR_peek_error, ERR_peek_last_error, ERR_get_error_line, ERR_peek_error_line, ERR_peek_last_error_line, ERR_get_error_line_data, ERR_peek_error_line_data, ERR_peek_last_error_line_data - obtain error code and data SYNOPSIS #include openssl/err.h unsigned long ERR_get_error(void); unsigned long ERR_peek_error(void); unsigned long ERR_peek_last_error(void); unsigned long ERR_get_error_line(const char **file, int *line); unsigned long ERR_peek_error_line(const char **file, int *line); unsigned long ERR_peek_last_error_line(const char **file, int *line); unsigned long ERR_get_error_line_data(const char **file, int *line, const char **data, int *flags); unsigned long ERR_peek_error_line_data(const char **file, int *line, const char **data, int *flags); unsigned long ERR_peek_last_error_line_data(const char **file, int *line , const char **data, int *flags); DESCRIPTION ERR_get_error() returns the earliest error code from the thread's error queue and removes the entry. This function can be called repeatedly until there are no more error codes to return. ERR_GET_LIB(3) OpenSSL ERR_GET_LIB(3) NAME ERR_GET_LIB, ERR_GET_FUNC, ERR_GET_REASON - get library, function and reason code SYNOPSIS #include openssl/err.h int ERR_GET_LIB(unsigned long e); int ERR_GET_FUNC(unsigned long e); int ERR_GET_REASON(unsigned long e); DESCRIPTION The error code returned by ERR_get_error() consists of a library num- ber, function code and reason code. ERR_GET_LIB(), ERR_GET_FUNC() and ERR_GET_REASON() can be used to extract these. The library number and function code describe where the error occurred, the reason code is the information about what went wrong. Each sub-library of OpenSSL has a unique library number; function and reason codes are unique within each sub-library. Note that different libraries may use the same value to signal different functions and rea- sons. -- /\ ASCII RIBBON NOTICE: If received in error, \ / CAMPAIGN Victor Duchovni please destroy and notify X AGAINST IT Security, sender. Sender does not waive / \ HTML MAILMorgan Stanley confidentiality or privilege, and use is prohibited. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
James A. Donald wrote: The correct mechanism is exception handling. Yes, I reckon there is a pretty wide consensus that exceptions provide a satisfactory solution to the sort of problems being discussed in this thread. If caller has provided a mechanism to handle the failure, that mechanism should catch the library generated exception. If the caller has provided no such mechanism, his program should terminate ungracefully. OK ... although I wouldn't make a virtue of doing things ungracefully. Unfortunately, there is no very portable support for exception handling in C. There is however support in C++, Corn, D, Delphi, Objective-C, Java, Eiffel, Ocaml, Python, Common Lisp, SML, PHP and all .NET CLS-compliant languages. That raises the question of whether mission-critical applications should be written in C. It is straightforward but laborious to simulate exception-throwing in C: extern int errno; /* try some stuff */ if (errno) return; /* return immediately on any error */ /* try some more stuff */ if (errno) return; /* return immediately on any error */ et cetera. This is laborious and inelegant, but no more so than the other gajillion things you need to do to provide any semblance of security in C, such as computing the (N) argument to strncat(,,N). In particular, if-errno-return is often much preferable to some other tricks that have recently been suggested, such as raising SIGABRT or passing a pointer to a function to be called when exceptional conditions are detected. The reason is that throwing an exception _pops_ the stack until a catcher if found, whereas signals and function-calls just push deeper into the stack ... they allow you to regain some control, but they don't make it easy to regain control _at the right place_. For completeness, let me say again that _exit() is just an exception that is caught by the parent process. If you are ever forced to deal with a package that exits when it shouldn't, it is straightforward to create a wrapper that does a fork, calls the package, and collects the exit status. That is, rather than: rslt = nasty(a, b); /* might trash the whole process */ we have: rslt = forku(nasty, a, b);/* better */ if (errno) return; But of course I hope you never have to face such a problem. If at all possible, use a language that supports exceptions. Absent exception handling, mission critical tasks should have no exceptions, which is best accomplished by the die-on-error standard. No, that is not the best way nor even a good way, let alone a standard. Halt on error is a tool for achieving error free code. Error free code is in fact achievable for really crucial applications. The more crucial the application, the more reason to write code that halts on error. Halting on every exceptional condition is like amputating to cure every headache. Keep in mind Dykstra's dictum: testing can perhaps show the presence of bugs, but testing can never show the absence of bugs. Exit-on-error is a _problem amplifier_. It guarantees that small problems detected during testing will not go unnoticed. But this is a very long way from being a guarantee of error-free code. a) Remember Dykstra's dictum. b) If you knew the code was error free, then by the Red Queen's logic you wouldn't even need to check for errors, and there would be no need for exit statements. The contrapositive is that if you put checks in the code, you are implicitly admitting that your code might not be error free. And there's the rub: you cannot possibly prove that during deployment (as opposed to during testing) using a _problem amplifier_ won't make things worse rather than better. Whatever happened to doing what's best for the customer? Doing what's most convenient for the programmer during testing, while making things worse for the customer during deployment ... that seems remarkably unprofessional. Last but not least, I object (again!) to the false dichotomy, i.e. the allegation that exceptional conditions must either a) result in an abort, or b) go undetected. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Tue, 14 Feb 2006 13:00:33 -0500, Steven M Bellovin said: Let me suggest a C-compatible possibility: pass an extra parameter to the library routines, specifying a procedure to call if serious errors occur. If that pointer is null, the library can abort. I agree. However the case at hand is a bit different. I can't imagine how any application or upper layer will be able to recover from that error (ENOENT when opening /dev/random). Okay, the special file might just be missing and a mknod would fix that ;-). Is it the duty of an application to fix an incomplete installation - how long shall this be taken - this is not the Unix philosophy. Salam-Shalom, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
In message [EMAIL PROTECTED], Werner Koch writes: On Tue, 14 Feb 2006 13:00:33 -0500, Steven M Bellovin said: Let me suggest a C-compatible possibility: pass an extra parameter to the library routines, specifying a procedure to call if serious errors occur. If that pointer is null, the library can abort. I agree. However the case at hand is a bit different. I can't imagine how any application or upper layer will be able to recover from that error (ENOENT when opening /dev/random). Okay, the special file might just be missing and a mknod would fix that ;-). Is it the duty of an application to fix an incomplete installation - how long shall this be taken - this is not the Unix philosophy. It can take context-specific error recovery. Maybe that's greying out the encrypt button on a large GUI. Maybe it's paging the system administrator. It can run 'mknod' inside the appropriate chroot partition, much as /sbin/init on some systems creates /dev/console. It can symlink /dev/geigercounter to /dev/random. It can load the kernel module that implements /dev/random. It can do a lot of things that may be more appropriate than exiting. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
-- John Denker wrote: Halting on every exceptional condition is like amputating to cure every headache. Keep in mind Dykstra's dictum: testing can perhaps show the presence of bugs, but testing can never show the absence of bugs. For truly critical applications, and I have written one such, there are better methods than testing. --digsig James A. Donald 6YeGpsZR+nOTh/cGwvITnSR3TdzclVpR0+pr3YYQdkG EQ0NuuGe3F81FVYLaVzuREVIM95sviNDw7cku0j6 4MEZw0qU0NMPYTNTSCMcjRi7wZSGRo06TUwlSmzr8 - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Tue, 14 Feb 2006 15:53:39 -0500, John Denker said: It is straightforward but laborious to simulate exception-throwing in C: extern int errno; /* try some stuff */ if (errno) return; /* return immediately on any error */ Except that this does not work. ERRNO gets set by most calls only on error so if everything went fine in the try ssome stuff you get random results. Shalom-Salam, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
Steven M. Bellovin wrote: In message [EMAIL PROTECTED], James A. Donald writes: -- Libgcrypt tries to minimize these coding errors; for example there are no error returns for the RNG - if one calls for 16 bytes of random one can be sure that the buffer is filled with 16 bytes of random. Now, if the environment is not okay and Libgcrypt can't produce that random - what shall we do else than abort the process. This way the errors will be detected before major harm might occur. I'm afraid I consider it instead a weakness in your API design that you have no way to indicate an error return from a function that may fail. The correct mechanism is exception handling. If caller has provided a mechanism to handle the failure, that mechanism should catch the library generated exception. If the caller has provided no such mechanism, his program should terminate ungracefully. Unfortunately, there is no very portable support for exception handling in C. There is however support in C++, Corn, D, Delphi, Objective-C, Java, Eiffel, Ocaml, Python, Common Lisp, SML, PHP and all .NET CLS-compliant languages. Absent exception handling, mission critical tasks should have no exceptions, which is best accomplished by the die-on-error standard. Precisely. I was preparing a post of my own, saying the same thing; you beat me to it. We all agree that critical errors like this should be caught; the only question is at what layer the action should take place. I'm an adherent to the Unix philosophy -- when a decision is made at a lower level, it takes away the ability of the higher level to do something different if appropriate, and this loss of flexibility is a bad thing. I have perhaps not been clear in some of my comments in this thread. I think there is a world of difference between critical errors and detecting internal inconsistency. In the case of inconsistency I claim that it is _never_ correct to continue running because every instruction executed is another potential hole for the attacker to use. Critical errors which do not indicate that something unexpected (as opposed to undesirable) has happened should, indeed, allow the caller to decide how they are handled. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
Werner Koch wrote: On Mon, 13 Feb 2006 03:07:26 -0500, John Denker said: Again, enough false dichotomies already! Just because error codes are open to abuse doesn't mean exiting is the correct thing to do. For Libgcrypt's usage patterns I am still convinced that it is the right decision. Then you should warn people that it is not safe to use in any high-privilege server application. cheers, DaveK -- Can't think of a witty .sigline today - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
-- John Denker wrote: Whatever happened to doing what's best for the customer? Doing what's most convenient for the programmer during testing, while making things worse for the customer during deployment ... that seems remarkably unprofessional. It is usually better for the customer that the program does nothing, than that it does something unexpected. This is particularly true in mission critical applications, such as for example a pace maker. Would you rather have an inactive pacemaker, or pacemaker busily doing something unexpected? In the case in question, going bad means that the program appears to be encrypting data, but is NOT encrypting data, or is only trivially encrypting data. This is far worse for the customer than an encryption program that simply aborts. Last but not least, I object (again!) to the false dichotomy, i.e. the allegation that exceptional conditions must either a) result in an abort, or b) go undetected. The correct solution to exceptional conditions is to use exceptions. This is not always practical or available, though it should be. The whole world should move to C++. If exceptions are not available, what then do we do? I say abort. --digsig James A. Donald 6YeGpsZR+nOTh/cGwvITnSR3TdzclVpR0+pr3YYQdkG yT/vxBNSRjFYGpU6iWTY1tvxDKTWkDa9wubFEmYD 40btwbJ8sjQGTu/vmkD4fjY1gud+1641iRf+Uq+Pb - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Sun, Feb 12, 2006 at 04:45:33PM +, Ben Laurie wrote: Werner Koch wrote: On Sat, 11 Feb 2006 12:36:52 +0100, Simon Josefsson said: 1) It invoke exit, as you have noticed. While this only happen in extreme and fatal situations, and not during runtime, it is not that serious. Yet, I agree it is poor design to do this in a library. I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. Quite so. No, libraries don't enough to decide what's fatal. The calling process (trying to an LDAP lookup via nsswitch.conf say...) may have other reasonable sources of data, and having the library kill it is unacceptable. If you try to resolve the problem by working around it will increase code complexity and thus error won't be detected. (Some systems might provide a failsafe mechanism at a top layer; e.g. by voting between independed developed code). But this is not why: if you attempt to fix impossible states, the problem is that you cannot know why (by definition) the code is in the state you are trying to fix, or what else might be broken. Continuing to run is giving the attacker the option to make good on his exploit. Not being able to access a resource is not an impossible state. Impossible states are corruption of internal data structures, invalid function arguments, ... Failure to obtain seed data is an error and needs to be reported as such. -- /\ ASCII RIBBON NOTICE: If received in error, \ / CAMPAIGN Victor Duchovni please destroy and notify X AGAINST IT Security, sender. Sender does not waive / \ HTML MAILMorgan Stanley confidentiality or privilege, and use is prohibited. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Mon, Feb 13, 2006 at 11:29:00AM +0100, Simon Josefsson wrote:
However, looking at the code, it is possible for Postfix to handle
this. They could have installed a log handler with libgcrypt, and
make sure to shut down gracefully if the log level is FATAL. The
recommendation to avoid GnuTLS because libgcrypt calls exit suggest
that the Postfix developers didn't care to investigate how to use
GnuTLS and libgcrypt properly. So I don't think there is any real
reason to change code in libgcrypt here. Postfix could be changed, if
they care about GnuTLS/libgcrypt.
Yeah, right, really easy when GnuTLS is called from the system LDAP
libraries... In any case the only way for the handler to avoid
process death is longjmp() to a context created before calling
GnuTLS/libgcrypt()... not a particularly robust solution.
void
_gcry_log_fatal( const char *fmt, ... )
{
va_list arg_ptr ;
va_start( arg_ptr, fmt ) ;
_gcry_logv( GCRY_LOG_FATAL, fmt, arg_ptr );
va_end(arg_ptr);
abort(); /* never called, but it makes the compiler happy */
}
the handler is invoked in _gcry_logv()... The Postfix TLS functionality
is built over OpenSSL (not GnuTLS) and OpenSSL has an error stack, which
the application can process as it sees fit. The libgrypt approach to
error reporting is not acceptable.
--
/\ ASCII RIBBON NOTICE: If received in error,
\ / CAMPAIGN Victor Duchovni please destroy and notify
X AGAINST IT Security, sender. Sender does not waive
/ \ HTML MAILMorgan Stanley confidentiality or privilege,
and use is prohibited.
-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
* Werner Koch: On Sat, 11 Feb 2006 12:36:52 +0100, Simon Josefsson said: 1) It invoke exit, as you have noticed. While this only happen in extreme and fatal situations, and not during runtime, it is not that serious. Yet, I agree it is poor design to do this in a library. I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. If you try to resolve the problem by working around it will increase code complexity and thus error won't be detected. (Some systems might provide a failsafe mechanism at a top layer; e.g. by voting between independed developed code). _exit in libraries is fine if you don't service multiple clients from a single process. However, with the advent of heavy VMs and stuff like that, there is a trend towards serving multiple clients from a single process (which is quite a bad idea in almost all cases, but this view is rather unpopular). There are also libraries which require proper cleanup procedures, otherwise the next program start can be quite costly (think of databases, where you want to avoid log replay). Some services have even been implemented following a single-process model for more than a decade (IRC servers, for example). A user-defined fatal error function (which must not return) would be a compromise, I think. Of course, such a function should never be called if you just see wrong or unsual input. But with a bit of optimism, the process could recover from an error which is not locally recoverable (throw an exception, terminate the offending thread, and leak the allocated resources). Now if the library maintains global, per-process state, this is a real problem. You can't know for sure if this state is consistent after a fatal error, unless you program carefully to avoid this situation. Yet another reason to move this functionality to a separate process. 8-) - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Sun, 12 Feb 2006 19:33:07 -0800 (PST), David Wagner said: Of course, it would be better for a crypto library to document this assumption explicitly than to leave it up to users to discover it the hard way, but I would not agree with the suggestion that this exit before Actually libgcrypt exactly does this. I have not looked at the postfix code under question but it sounds like stderr has beend duped to /dev/null and no log handler has been registered (e.g. to divert logging to syslog). Shalom-Salam, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Sun, 12 Feb 2006 23:57:42 -, Dave Korn said: :-) Then what was EINVAL invented for? [ Then for what was assert invented for? ] Really it's never ok for anything, not even games, and any program that fails to check error return values is simply not properly coded, full stop. I agree. But the reality is not that of the text books. But abort()-ing in a library is also a big problem, because it takes control away from the main executable. That can be a massive security vulnerability on Windows. If you can get a SYSTEM-level service that Huh? According to ISO C and POSIX abort raises SIGABRT and the default action is abnormal *process termination* - if your view is that process termination takes away control from the main executable I wonder how a file can control a process (unless the kernels plays nasty games with on demand paging). To my limited Windows experience abort() does terminate the process. I have ported quite some Unix applications nativly to Windows and never got in semantic problems you describe. Anyway, Windows is strange (atexit lists per DLL and such) but Libgcrypt is not really supported there. ... receive request from client ... fail to service it because libgcrypt returns errors.. return error to caller ... rather than for it to abort. Being in an insane state libgcrypt can't assure that this main loop will continue to run - the stack might already be corrupted. We don't know and thus assert(!fubar). I'm afraid I consider it instead a weakness in your API design that you have no way to indicate an error return from a function that may fail. By design there can't be any error. If there is an error something really strange has occured, like improper chrooting. Perhaps libgcrypt could call abort in debug builds and return error codes in production builds? Your joking right? I am usually quite sure that no attacker has made it to one of the machines used for debugging. Outside in the Internet wilderness I should then switch off all protection? That is like wearing a hard hat in bed and take it off at the construction site. Salam-Shalom, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Mon, 13 Feb 2006 11:29:00 +0100, Simon Josefsson said: That /dev/random doesn't exist seem like a quite possible state to me. Running Linux this is not possible because /dev/random is guarenteed to be available. Further, a library is not in a good position to report errors. A users will sit there wondering why Postfix, or some other complex I don't know where Postfix dumps the error messages from Libcrypt: fd = open( name, O_RDONLY ); if( fd == -1 ) log_fatal (can't open %s: %s\n, name, strerror(errno) ); I guess you need to blame postfix for this. recommendation to avoid GnuTLS because libgcrypt calls exit suggest that the Postfix developers didn't care to investigate how to use GnuTLS and libgcrypt properly. So I don't think there is any real So may I conclude that it is actually Good Thing that in this case libgcrypt refrained from continuing to preserve the caller from false security. I'd say that the most flexible approach for a library is to write thread-safe code that doesn't need access to mutexes to work properly. Yes. We discussed this already at length at more appropriate places. That seem like a poor argument to me. It may be valid for embedded devices, but for most desktop PCs, Linux should provide a useful /dev/urandom. I can only tell what Ted told me years ago. Shalom-Salam, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Mon, 13 Feb 2006 03:07:26 -0500, John Denker said: That might lead to an argument in favor of exceptions instead of error codes, along the following lines: -- Naive code doesn't catch the exception. However (unlike returned error codes) this does not cause the exception to be lost. -- The exception percolates up the call-tree until it is caught by some non-naive code (if any). -- If all the code is naive, then the uncaught exception terminates the process ... to the delight of the exit on error faction. However (!!!) unlike a plain old exit, throwing an exception leaves the door open for non-naive code to implement a nuanced response to the exceptional condition. Actually the plain C similar thing is done for an internal error: SIGABRT is raised and the top level code (or in theory any layer in between) may catch it and try to continue. Okay, this won't work in practise because signal handling between independent developed code (libraries) is guaranteed not to work correctly. And yes, we need to discuss whether whether a failed open should abort or exit. As of now it does an exit and not an abort() but I won't insist on this. Again, enough false dichotomies already! Just because error codes are open to abuse doesn't mean exiting is the correct thing to do. For Libgcrypt's usage patterns I am still convinced that it is the right decision. Salam-Shalom, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
| I disagree strongly here. Any code which detects an impossible state | or an error clearly due to a programming error by the caller should | die as soon as possible. | | That is a remarkably unprofessional suggestion. I hope the people | who write software for autopilots, pacemakers, antilock brakes, | etc. do not follow this suggestion. | | This just shows the dangers of over-generalization. And *this* shows the danger of false dichotomies. | Of course, we have to decide which is more important: integrity, | or availability. I suspect that in the overwhelming majority (perhaps | all) of the cases where libgcrypt is used, integrity is more important | than availability. If that is true, well, if in doubt, it's better to | fail closed than to fail open. | | You rightly points out that there are important applications where | availability is more important than integrity. However, I suspect | those cases are not too common when building Internet-connected desktop | applications. A library can't possibly know what kind of applications it will be part of! | I think the attitude that it's better to die than to risk letting an | attacker take control of the crypto library is defensible, in many cases. | Of course, it would be better for a crypto library to document this | assumption explicitly than to leave it up to users to discover it the | hard way, but I would not agree with the suggestion that this exit before | failing open stance is always inappropriate. No, the library thinks it can call exit() is *always* inappropriate. There are reasonable ways to deal with this kind of thing that are just as safe, but allow general-purpose use. For example: 1. On an error like this, put the encrypted connection (or whatever it is) into a permanent error state. Any further calls act as if the connection had been closed. Any incoming or outgoing data is erased and discarded. Any keying material is immediately erased and discarded. Of course, return error statuses to the caller appropriately. (You don't return error statuses? Then you're already talking about a poor design. Note that there's a world of difference between returning an error status *locally* and sending it over the wire. The latter can turn your code into an oracle. The former ... well, unless you're writing a closed-source library for a secret protocol and you assume your code and protocol can't be reverse-engineered, the local user can *always* get this information somehow.) 2. When such an error occurs, throw an exception. In a language that supports exceptions as such (C++, Java), use the native mechanism. For languages that don't support exceptions, you can call a function through a pointer. By default, the function can call, or simply be, exit(); but the user can specify his own function. The function *must* be allowed to do something other than call exit()! In general, this technique has to be combined with technique 1. Granted, a user *could* write code that leaked important information upon being informed of an error. But he would have to try. And, frankly, there's not a damn thing you can do to *prevent* that. Most Unix systems these days allow you to interpolate functions over standard library functions. You think you're calling exit(), or invoking kill()? Hah, I've replaced them with my own functions. So there. (No interpolation? Patching compiled code to change where a function call goes is pretty easy.) Of course, all this is nonsensical for an open-source library anyway! You're kidding yourself if you think *any* programming practice will protect you against a programmer who needs his program to do something that you consider a bad idea. But the whole approach is fundamentally wrong-headed. The user of your library is *not* your enemy. You should be cooperating with him, not trying to box him in. If you treat him as your enemy, he'll either choose another library - or find a way to work around your obstinacy. -- Jerry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
-- Werner Koch retorted: I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. John Denker wrote: That is a remarkably unprofessional suggestion. I hope the people who write software for autopilots, pacemakers, antilock brakes, etc. do not follow this suggestion. If bad code halts, it will not get incorporated into production code. If bad code produces ignored error messages, it will get incorporated into production code, including pacemakers etc. Therefore libraries intended for use with pacemakers, anti lock brakes, and the like, should die on error (which in the case of antilock brakes forces a hard reboot. Code intended for pacemakers and the like should be error free. If you write libraries intended to continue after error, you are writing on the assumption that the pacemaker code will be buggy, and we don't really care, we are going to ship it anyway, bugs and all into other people's chests. People who write code for pacemakers that continues on error should be shot. Halt on error is a tool for achieving error free code. Error free code is in fact achievable for really crucial applications. The more crucial the application, the more reason to write code that halts on error. --digsig James A. Donald 6YeGpsZR+nOTh/cGwvITnSR3TdzclVpR0+pr3YYQdkG Cau3evB8n2DnP2D8ej3FHKKnKnMeseK65pUDF346 4FbXJRaadlYWOfMnkhNKfdLxDaKNb58AoLBUm8ox9 - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
-- Libgcrypt tries to minimize these coding errors; for example there are no error returns for the RNG - if one calls for 16 bytes of random one can be sure that the buffer is filled with 16 bytes of random. Now, if the environment is not okay and Libgcrypt can't produce that random - what shall we do else than abort the process. This way the errors will be detected before major harm might occur. I'm afraid I consider it instead a weakness in your API design that you have no way to indicate an error return from a function that may fail. The correct mechanism is exception handling. If caller has provided a mechanism to handle the failure, that mechanism should catch the library generated exception. If the caller has provided no such mechanism, his program should terminate ungracefully. Unfortunately, there is no very portable support for exception handling in C. There is however support in C++, Corn, D, Delphi, Objective-C, Java, Eiffel, Ocaml, Python, Common Lisp, SML, PHP and all .NET CLS-compliant languages. Absent exception handling, mission critical tasks should have no exceptions, which is best accomplished by the die-on-error standard. --digsig James A. Donald 6YeGpsZR+nOTh/cGwvITnSR3TdzclVpR0+pr3YYQdkG Ywzx2XsxbvPNX+eeGZVUpnq16108eQo1eBvq8K1I 46HVM7avhGKHTF4Y1SqhFSUdIsTlbJvpXX43jkvQP - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Tue, Feb 14, 2006 at 12:44:39PM +1000, James A. Donald wrote: Absent exception handling, mission critical tasks should have no exceptions, which is best accomplished by the die-on-error standard. Absent good library design, the developer's goals are best accomplished with the roll-your-own standard. If the authors of libgrypt instead of saying sorry, we know, it is a difficult problem, we are working on it, instead become defensive and erect false dichotomies to defend the developer from his own folly, I can add libgrypt to my list of tools to avoid when building large systems. As I said before, Postfix does not use GnuTLS directly, rather it is sometimes a victim of libgrypt design via GnuTLS imbedded in the system LDAP library. The current libgrypt is IMHO not suitable for linking into LDAP libraries, database client-server communication libraries, SMTP servers... As for Postfix, it does entropy gathering out-of-process (in the tlsmgr(8) daemon). The SMTP server and client daemons get entropy indirectly from tlsmgr(8) to seed their internal PRNG. Postfix uses OpenSSL, and error conditions in OpenSSL are recoverable (Postfix can and will return 454 in response to STARTTLS, fatal errors are not appropriate in this context). Postfix makes use of error reporting hooks in MySQL, PgSQL, SASL, OpenSSL, (non-GnuTLS) OpenLDAP... none of these have been reported to abruptly terminate the calling process instead of reporting errors to the caller. -- /\ ASCII RIBBON NOTICE: If received in error, \ / CAMPAIGN Victor Duchovni please destroy and notify X AGAINST IT Security, sender. Sender does not waive / \ HTML MAILMorgan Stanley confidentiality or privilege, and use is prohibited. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
In message [EMAIL PROTECTED], James A. Donald writes: -- Libgcrypt tries to minimize these coding errors; for example there are no error returns for the RNG - if one calls for 16 bytes of random one can be sure that the buffer is filled with 16 bytes of random. Now, if the environment is not okay and Libgcrypt can't produce that random - what shall we do else than abort the process. This way the errors will be detected before major harm might occur. I'm afraid I consider it instead a weakness in your API design that you have no way to indicate an error return from a function that may fail. The correct mechanism is exception handling. If caller has provided a mechanism to handle the failure, that mechanism should catch the library generated exception. If the caller has provided no such mechanism, his program should terminate ungracefully. Unfortunately, there is no very portable support for exception handling in C. There is however support in C++, Corn, D, Delphi, Objective-C, Java, Eiffel, Ocaml, Python, Common Lisp, SML, PHP and all .NET CLS-compliant languages. Absent exception handling, mission critical tasks should have no exceptions, which is best accomplished by the die-on-error standard. Precisely. I was preparing a post of my own, saying the same thing; you beat me to it. We all agree that critical errors like this should be caught; the only question is at what layer the action should take place. I'm an adherent to the Unix philosophy -- when a decision is made at a lower level, it takes away the ability of the higher level to do something different if appropriate, and this loss of flexibility is a bad thing. As noted, the best answer is a modern language that supports exceptions. (Sorry, SIGABRT and setjmp/longjmp just don't cut it.) Let me suggest a C-compatible possibility: pass an extra parameter to the library routines, specifying a procedure to call if serious errors occur. If that pointer is null, the library can abort. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
David Wagner wrote: This just shows the dangers of over-generalization. One could make an even stronger statement about the dangers of making assumptions that are not provably correct. Of course, we have to decide which is more important: integrity, or availability. That is a false dichotomy. I suspect that in the overwhelming majority (perhaps all) of the cases where libgcrypt is used, integrity is more important than availability. If that is true, well, if in doubt, it's better to fail closed than to fail open. Again: False dichotomy is a fallacy, and has been recognized for 2000 years as such. Showing that one extreme is bad does not prove that the opposite extreme is good. The whole point of my previous note was to argue for more nuanced handling. Progressing from one knee-jerk handing to a choice between two knee-jerk handlings is not much progress. I think the attitude that it's better to die than to risk letting an attacker take control of the crypto library is defensible, in many cases. Again, that's the wrong question; it's not an either/or proposition. We can agree that letting the attacker take control of the situation is a Bad Thing, but it is preposterous to think that exiting is provably correct in all situations where the library might be put to use. It is just plain arrogant for low-level code to arrogate to itself a decision that rightfully belongs to higher-level code. Werner Koch wrote in part: Sure, for many APIs it is posssible to return an error code but this requires that the caller properly checks error codes. We have all seen too many cases were return values are not checked and the process goes ahead assuming that everything went well That is narrowly true as stated, but it does not prove that exiting is the correct thing to do. That might lead to an argument in favor of exceptions instead of error codes, along the following lines: -- Naive code doesn't catch the exception. However (unlike returned error codes) this does not cause the exception to be lost. -- The exception percolates up the call-tree until it is caught by some non-naive code (if any). -- If all the code is naive, then the uncaught exception terminates the process ... to the delight of the exit on error faction. However (!!!) unlike a plain old exit, throwing an exception leaves the door open for non-naive code to implement a nuanced response to the exceptional condition. Again, enough false dichotomies already! Just because error codes are open to abuse doesn't mean exiting is the correct thing to do. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
Werner Koch [EMAIL PROTECTED] writes: On Sat, 11 Feb 2006 12:36:52 +0100, Simon Josefsson said: 1) It invoke exit, as you have noticed. While this only happen in extreme and fatal situations, and not during runtime, it is not that serious. Yet, I agree it is poor design to do this in a library. I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. If you try to resolve the problem by working around it will increase code complexity and thus error won't be detected. (Some systems might provide a failsafe mechanism at a top layer; e.g. by voting between independed developed code). That /dev/random doesn't exist seem like a quite possible state to me. The application would want to shut down gracefully when the library detect that condition. The application may be processing files in different threads. Further, a library is not in a good position to report errors. A users will sit there wondering why Postfix, or some other complex application died, without any clues. Returning an error and providing a foo_strerror() function at least make it possible to report a useful error to the user. I would agree if we are only talking about truly fatal cases, like asserts() to check explicit pre-conditions for a function, but I disagree when we move into the area if easily anticipated problems. However, looking at the code, it is possible for Postfix to handle this. They could have installed a log handler with libgcrypt, and make sure to shut down gracefully if the log level is FATAL. The recommendation to avoid GnuTLS because libgcrypt calls exit suggest that the Postfix developers didn't care to investigate how to use GnuTLS and libgcrypt properly. So I don't think there is any real reason to change code in libgcrypt here. Postfix could be changed, if they care about GnuTLS/libgcrypt. It is the same rationale why defining NDEBUG in production code is a Bad Thing. Agreed. 2) If used in a threaded environment, it wants to have access to thread primitives. The primary reason was for RNG pool locking (where it is critical), but I think the primitives are now used in other places too. GnuTLS is thread agnostic, so it can't initialize libgcrypt properly. Against our advise Nikos rejected to implement a proper initialization. Libraries and threading is actually a deep problem. It usually works well on GNU/Linux systems but this is more of coincidence than by design. We did quite some research on this and experimented with different ways of automagically initializing the thread primitives correctly; they all fail either at runtime or create headaches when trying to write proper build rules. The current approach is by far the most flexible and safest. But yes, the fact that one library needs an initialization can't be hidden from the application even if the application is using the lib only indirectly (Foo-OpenLDAP-GnuTLS-Libgcrypt). I'd say that the most flexible approach for a library is to write thread-safe code that doesn't need access to mutexes to work properly. Implementing the RNG functions like this is a challenge, and may require kernel-level support (see below), but giving up and requiring thread hooks seem sub-optimal. list. I think the Linux /dev/urandom implementation is sub-optimal. This is known since Ted Ts'o wrote /dev/random and justified by requirement of the Linux hackers to keep the memory use by a minimal Linux build low. That seem like a poor argument to me. It may be valid for embedded devices, but for most desktop PCs, Linux should provide a useful /dev/urandom. It seems that it would be possible to write a new /dev/*random implementation that is more useful by libgcrypt and other RNG libraries. Thanks, Simon - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
Werner Koch wrote: On Sat, 11 Feb 2006 12:36:52 +0100, Simon Josefsson said: 1) It invoke exit, as you have noticed. While this only happen in extreme and fatal situations, and not during runtime, it is not that serious. Yet, I agree it is poor design to do this in a library. I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. Quite so. If you try to resolve the problem by working around it will increase code complexity and thus error won't be detected. (Some systems might provide a failsafe mechanism at a top layer; e.g. by voting between independed developed code). But this is not why: if you attempt to fix impossible states, the problem is that you cannot know why (by definition) the code is in the state you are trying to fix, or what else might be broken. Continuing to run is giving the attacker the option to make good on his exploit. Sure, for many APIs it is posssible to return an error code but this requires that the caller properly checks error codes. Again, this it not the issue. If you continue to run code you are running it against an unknown state of the code and data. You are allowing more leeway for exploits. This is true even if the code is just printing a warning (don't forget that to print a warning you must typically execute a substantial body of library and system code). We have all seen too many cases were return values are not checked and the process goes ahead assuming that everything went well - this might be okay for games but definitely not for cryptographic applications. Libgcrypt tries to minimize these coding errors; for example there are no error returns for the RNG - if one calls for 16 bytes of random one can be sure that the buffer is filled with 16 bytes of random. Now, if the environemnt is not okay and Libgcrypt can't produce that random - what shall we do else than abort the process. This way the errors will be detected before major harm might occur. It is the same rationale why defining NDEBUG in production code is a Bad Thing. I agree. Cheers, Ben. -- http://www.links.org/ - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
On Sun, 12 Feb 2006 13:46:05 -0500, John Denker said: That is a remarkably unprofessional suggestion. I hope the people who write software for autopilots, pacemakers, antilock brakes, etc. do not follow this suggestion. Thus my remark about a independend failsafe system. I strongly hope that for life critical systems nobody even things about throwing in a bunch of general purpose libraries and declares the task as done. Fortunately these systems have resource constraints so that such a solution won't come to mind anyway. First of all, impossible is the wrong word. If the condition s,impossible,not foreseen/tested/coded by the developer, previous tick's tasks are finished. What do you do, exit? If Yes. And one of the concurrent running system will take over. In 1969 this system used to be Armstrong, though. There are other stories like this, including funny (?) stories of what happens if exceptions are not handled so well. Terminating a process is a well handled exception. Think of hardware failure; continue while knowing that tehre is something really weird going on?? More generally: library routines should never exit. They almost If you are thinking of exit please mentally translate this to assert. Still believing one should never call assert(0) in a library? Nitpickers note: I can imagine a situation where the stack is so messed up that you can't thrown an exception or even return from Die as soon as you can; kill (getpid(),SIGKILL) might even be justified in such a situation. Try to make an attackers life as hard as possible. Yes, for life critical systems an attack scenario might be different to judge and thus the design needs to be different. Obviously I agree with Ben that that there is no way to know the current state if something went wrong in unexpected ways. Shalom-Salam, Werner - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: GnuTLS (libgrypt really) and Postfix
Werner Koch wrote: On Sat, 11 Feb 2006 12:36:52 +0100, Simon Josefsson said: 1) It invoke exit, as you have noticed. While this only happen in extreme and fatal situations, and not during runtime, it is not that serious. Yet, I agree it is poor design to do this in a library. I disagree strongly here. Any code which detects an impossible state or an error clearly due to a programming error by the caller should die as soon as possible. :-) Then what was EINVAL invented for? Sure, for many APIs it is posssible to return an error code but this requires that the caller properly checks error codes. We have all seen too many cases were return values are not checked and the process goes ahead assuming that everything went well - this might be okay for games but definitely not for cryptographic applications. Really it's never ok for anything, not even games, and any program that fails to check error return values is simply not properly coded, full stop. But abort()-ing in a library is also a big problem, because it takes control away from the main executable. That can be a massive security vulnerability on Windows. If you can get a SYSTEM-level service that listens on a well known pipe or LPC port to abort(), you can often steal it's pipe or port and escalate your privileges It would be far preferable for the service to remain running in a main loop that ends up operating as ... ... receive request from client ... fail to service it because libgcrypt returns errors.. return error to caller ... rather than for it to abort. Libgcrypt tries to minimize these coding errors; for example there are no error returns for the RNG - if one calls for 16 bytes of random one can be sure that the buffer is filled with 16 bytes of random. Now, if the environemnt is not okay and Libgcrypt can't produce that random - what shall we do else than abort the process. This way the errors will be detected before major harm might occur. I'm afraid I consider it instead a weakness in your API design that you have no way to indicate an error return from a function that may fail. It is the same rationale why defining NDEBUG in production code is a Bad Thing. Perhaps libgcrypt could call abort in debug builds and return error codes in production builds? cheers, DaveK -- Can't think of a witty .sigline today - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
