Re: gettimeofday(2) does not conform to POSIX.1-2008?
On Sun, 3 Dec 2017, Theo de Raadt wrote: > > EFAULT is kinda interesting in that POSIX documents its existence but > > does not require it to ever be returned. Here's what POSIX says > > normatively: ... > POSIX is a wimpy subset of UNIX, specified so unstrictly that it > becomes difficult to build secure software upon the foundation. Sure, but I wouldn't point to its (non-)specification of EFAULT as part of that. > I believe if you don't teach people the need for strict careful > specifications -- by example -- then don't be surprised they don't learn > the lessons elsewhere, and their results are chaotic and fragile. Even when standards specify strict behavior, getting a sufficient fraction of the ecosystem to be on implementations that actually _do_ that, sufficient to have the desired effect on software development practice, is something we--the software economy--have not figured out how to accomplish. The incentive structures push the wrong way: first-to-market gains, risk and cost shifting that reduce the downside of insecure or non-compliant implementations, etc. To fix the ecosystem, the flow of benefits will stop favoring those who take shortcuts in security/safety/compliance. Philip
Re: gettimeofday(2) does not conform to POSIX.1-2008?
> EFAULT is kinda interesting in that POSIX documents its existence but does > not require it to ever be returned. Here's what POSIX says normatively: > [EFAULT] > Bad address. The system detected an invalid address in attempting to > use an argument of a call. The reliable detection of this error > cannot be guaranteed, and when not detected may result in the > generation of a signal, indicating an address volation, which is > sent to the process. > > _Informatively_ (no compliance requirement!) it says this in the > rationale: > POSIX.1 requires (in the ERRORS sections of function descriptions) > certain error values to be set in certain conditions because many > existing applications depend on them. Some error numbers, such > as [EFAULT], are entirely implementation-defined and are noted > as such in their description in the ERRORS section. This section > otherwise allows wide latitude to the implementation in handling > error reporting. > ... > [EFAULT] Most historical implementations do not catch an error > and set errno when an invalid address is given to the > functions wait(), time(), or times(). Some implementations > cannot reliably detect an invalid address. And most > systems that detect invalid addresses will do so only > for a system call, not for a library routine. POSIX is a wimpy subset of UNIX, specified so unstrictly that it becomes difficult to build secure software upon the foundation. I believe if you don't teach people the need for strict careful specifications -- by example -- then don't be surprised they don't learn the lessons elsewhere, and their results are chaotic and fragile. the lack of a strict decision is a decision to be sloppy. sand. too much is built on sand.
Re: gettimeofday(2) does not conform to POSIX.1-2008?
Standards state a contract or implication: *if* you, the developer follow _these_ rules, *then* the standardized item will follow _these_other_ rules (if compliant). If you violate the implication, then the standard no longer applies and you have lost the guarantees of the standard. _If_ you build an electrical plug with prongs of a certain composition, shape, size, and distance apart and nothing projecting out of a described volume, _then_ it will fit into this standard plug with sufficient contact to carry a particular current and resist physical force to move/remove it of some size. If you go make the prongs of the wrong shape or size, then the rules are off and it might fail in many different ways: wrong shape or size? Might not make contact enough to carry the standard current, or might not resist being removed sufficiently. Too close together? Might not fit at all or might break when inserted. etc. In the case of POSIX, there's a long intro which (tries to) set out the rules that you have to follow. Some of these are effectively compile-time restrictions (e.g., you MUST NOT define macros, variables, or functions with various names) and others which are effectively run-time restrictions (e.g., for many functions, particular arguments MUST be valid C strings). If you go outside those then, again, the rules are off. If you put "#define struct union" before "#include ", or pass a NULL pointer as the first argument to gettimeofday(2) then you've violated the application side of the POSIX contract and the system is not required to do anything in particular. Yes, standards can be written to require graceful degradation, effectively saying "if you don't follow _all_ the rules but did follow at least this shorter list, then the standardized item must still keep up part of its side." This is common in security/safety standards. In POSIX this shows up in some interfaces being required to return EINVAL or similar in certain cases...but POSIX DOES NOT specify every possible error case and explicitly leaves many such cases undefined, leaving it to applications to do whatever meets their goal, be it speed, simplicity, debugability, etc. So, background having been set: On Sat, 2 Dec 2017, Scott Cheloha wrote: > TL;DR: POSIX.1-2008 says gettimeofday(2) can't return anything but 0, > but we return -1 on invalid input. We claim that our gettimeofday(2) > conforms to POSIX.1-2008. Is something off here? Nope. What POSIX says is that *if you follow the POSIX rules*, then gettimeofday() will only return zero. It's perfectly acceptable for a particular implementation to document its own behavior above and beyond what the standard requires. We happen to document that if you pass a bogus pointer to gettimeofday(2), something not permitted by POSIX, then it'll fail with error EFAULT. EFAULT is kinda interesting in that POSIX documents its existence but does not require it to ever be returned. Here's what POSIX says normatively: [EFAULT] Bad address. The system detected an invalid address in attempting to use an argument of a call. The reliable detection of this error cannot be guaranteed, and when not detected may result in the generation of a signal, indicating an address volation, which is sent to the process. _Informatively_ (no compliance requirement!) it says this in the rationale: POSIX.1 requires (in the ERRORS sections of function descriptions) certain error values to be set in certain conditions because many existing applications depend on them. Some error numbers, such as [EFAULT], are entirely implementation-defined and are noted as such in their description in the ERRORS section. This section otherwise allows wide latitude to the implementation in handling error reporting. ... [EFAULT] Most historical implementations do not catch an error and set errno when an invalid address is given to the functions wait(), time(), or times(). Some implementations cannot reliably detect an invalid address. And most systems that detect invalid addresses will do so only for a system call, not for a library routine. ... > Does this make our implementation non-conforming and does something > in the manpage need to change? > > I do note that gettimeofday(2) predates [3] POSIX (and me :p), so > perhaps this is just an ancient compromise between BSD and the > standards committees? It's not so much a compromise as the standards committee making a decision about where to draw the line of covered-vs-not. In theory, a standard could be written that exactly described all the error cases...but it would take forever to write and resolve the ambiguities in the description and no implementation would ever comply with all the details. And what would the benefit be? We don't _want_ application authors to use those corners! So, since it doesn't seem to be useful to
Re: gettimeofday(2) does not conform to POSIX.1-2008?
On Sat, Dec 02, 2017 at 11:54:23AM -0700, Theo de Raadt wrote: > > > Wouldn't be the first error in POSIX. The documents have gotten > > > sloppier. I think this is due to their commitee not acquiring any > > > sensible young blood. > > > > fwiw, the "return zero and only zero" behavior traces back at least > > as far as here (SUSv2, I think): > > > > http://pubs.opengroup.org/onlinepubs/7908799/xsh/gettimeofday.html > > > > so if it's an error it isn't a "new" error. > > Hang on, you say behavior. > > Behavior is a quality of implementation. If you research back to the > dawn of time, gettimeofday would return -1 and EFAULT because it uses > a kernel function copyout(9) to do it's work, or maybe even earlier > a pair of suword calls with the same error return. > > Implementation leads to that behavior. > > Did someone do it differently? Maybe, but developers need to know > that passing an invalid pointer can result in an error return rather > than some weird fault. > > The POSIX document is a descriptive document. It should describe the > existing behavior to enhance compatibility. There are strategies for > how one may describe a behavior: minimally or maximally, with impacts > on readership in all directions. This instance shows they prefer to > be minimally descriptive or dare I say it -- inaccurate by omission. > They may have some goal in mind, but it still is inaccurate, and their > justifications are kept largely secret. It is a secretive cabal, and > don't point me to austin-group lists, it is nothing more than lipstick > on a pig. > > When descriptive documents willfully describe a system incompletely, > it makes it harder for software developers to consider them as a > proscriptive or leading agency. Errors result. How apropos that I write "behavior" when I meant "wording." Point well taken. -- Scott Cheloha
Re: gettimeofday(2) does not conform to POSIX.1-2008?
> > Wouldn't be the first error in POSIX. The documents have gotten > > sloppier. I think this is due to their commitee not acquiring any > > sensible young blood. > > fwiw, the "return zero and only zero" behavior traces back at least > as far as here (SUSv2, I think): > > http://pubs.opengroup.org/onlinepubs/7908799/xsh/gettimeofday.html > > so if it's an error it isn't a "new" error. Hang on, you say behavior. Behavior is a quality of implementation. If you research back to the dawn of time, gettimeofday would return -1 and EFAULT because it uses a kernel function copyout(9) to do it's work, or maybe even earlier a pair of suword calls with the same error return. Implementation leads to that behavior. Did someone do it differently? Maybe, but developers need to know that passing an invalid pointer can result in an error return rather than some weird fault. The POSIX document is a descriptive document. It should describe the existing behavior to enhance compatibility. There are strategies for how one may describe a behavior: minimally or maximally, with impacts on readership in all directions. This instance shows they prefer to be minimally descriptive or dare I say it -- inaccurate by omission. They may have some goal in mind, but it still is inaccurate, and their justifications are kept largely secret. It is a secretive cabal, and don't point me to austin-group lists, it is nothing more than lipstick on a pig. When descriptive documents willfully describe a system incompletely, it makes it harder for software developers to consider them as a proscriptive or leading agency. Errors result.
Re: gettimeofday(2) does not conform to POSIX.1-2008?
On Sat, Dec 02, 2017 at 11:05:17AM -0700, Theo de Raadt wrote: > > TL;DR: POSIX.1-2008 says gettimeofday(2) can't return anything but 0, > > but we return -1 on invalid input. > > It must return -1 to indicate the tp pointer is to invalid memory. > All system calls indicate invalid pointer arguments in that way, and > this one is not special. > > [EFAULT] An argument address referenced invalid memory. > > > We claim that our gettimeofday(2) conforms to POSIX.1-2008. Is > > something off here? > > POSIX appears to be wrong. > > Yes there are people doing VDSO things, which may not be able to > indicate a fault, and the process will crash instead when it fondles > it's own missing memory. But I don't see any way that gives the > commitee latitude to remove historic behaviour. > > > Also, gettimeofday(2) is an XSI extension, so I think at a > > minimum the STANDARDS section needs to be tweaked (attached). > > As a general rule we don't speak of XSI unless it exposes programmers > to problems. Ah, okay. > Mentioning XSI in the way you propose doesn't actually > help anyone write better code since noone knows what XSI is. So in > this case, I don't think XSI should be mentioned, the words perform > no educational function. Alrighty. > > Long version: > > > > POSIX.1-2008 [1] says: > > > > > The gettimeofday() function shall return 0 and no value shall > > > be reserved to indicate an error. > > Wouldn't be the first error in POSIX. The documents have gotten > sloppier. I think this is due to their commitee not acquiring any > sensible young blood. fwiw, the "return zero and only zero" behavior traces back at least as far as here (SUSv2, I think): http://pubs.opengroup.org/onlinepubs/7908799/xsh/gettimeofday.html so if it's an error it isn't a "new" error. > > Does this make our implementation non-conforming and does something > > in the manpage need to change? > > I don't think anything should change. Okay. Appreciate the quick reply! -- Scott Cheloha
Re: gettimeofday(2) does not conform to POSIX.1-2008?
> TL;DR: POSIX.1-2008 says gettimeofday(2) can't return anything but 0, > but we return -1 on invalid input. It must return -1 to indicate the tp pointer is to invalid memory. All system calls indicate invalid pointer arguments in that way, and this one is not special. [EFAULT] An argument address referenced invalid memory. > We claim that our gettimeofday(2) conforms to POSIX.1-2008. Is > something off here? POSIX appears to be wrong. Yes there are people doing VDSO things, which may not be able to indicate a fault, and the process will crash instead when it fondles it's own missing memory. But I don't see any way that gives the commitee latitude to remove historic behaviour. > Also, gettimeofday(2) is an XSI extension, so I think at a > minimum the STANDARDS section needs to be tweaked (attached). As a general rule we don't speak of XSI unless it exposes programmers to problems. Mentioning XSI in the way you propose doesn't actually help anyone write better code since noone knows what XSI is. So in this case, I don't think XSI should be mentioned, the words perform no educational function. > Long version: > > POSIX.1-2008 [1] says: > > > The gettimeofday() function shall return 0 and no value shall > > be reserved to indicate an error. Wouldn't be the first error in POSIX. The documents have gotten sloppier. I think this is due to their commitee not acquiring any sensible young blood. > Does this make our implementation non-conforming and does something > in the manpage need to change? I don't think anything should change.
