On Tue, Dec 06, 2016 at 02:50:23PM +0100, Michael Kerrisk wrote:
> Hello all,
>
> Following a discussion [4][5] (and a bit of education for me, thanks
> to Rich Felker) on the GNU C Library (libc-alpha) mailing list, I have
> some questions about bug 529 [0].
>
> 1. If I understand correctly, the proposed changes for Issue 8 will in
> effect declare to be broken those existing applications that treat
> EINTR from close() as "the file is closed". (I.e., per the proposed
> changes [7], implementations will not be allowed to have close() fail
> with EINTR while also closing the file.) However, those existing
> applications are behaving correctly in terms of what the
> implementations do. Is my understanding of the intentions of the Issue
> 8 changes correct? If yes, I presume the path is that such
> applications would be conformant to older POSIX, but won't be
> conformant to Issue 8. (Right?)
I don't think the change affects application conformance, since
applications that rely on either behavior prior to Issue 8 are relying
on something that's *unspecified* (not even implementation-defined).
An application that assumes the fd might have been closed, just to be
safe, is conforming before and after the Issue 8 change, but
definitely does leak file descriptors on EINTR under an implementation
conforming to Issue 8, whereas it may or may not leak on an
implementation conforming only to Issue 7 or earlier.
Where conformance is affected is for implementations. An
implementation that never fails with EINTR conforms to both Issue 8
and earlier issues (as long as it does not remain blocked in close
when hit by an interrupting signal; that would be non-conforming, I
think). An implementation that leaves the fd open when failing with
EINTR also conforms to both Issue 8 and earlier issues. But an
implementation (like Linux kernel without userspace fixups) that
closes the fd when failing with EINTR is conforming only to Issue 7
and earlier; it's nonconforming to Issue 8.
> 2. From the bug thread, I've missed something. I understand the
> argument that EINTR was being treated inconsistently for close() on
> Linux and other systems, but the thing is that implementations with
> the close() "EINTR means the file descriptor is closed" behavior
> exist, and seem even to be common. Why (given the variation in
> existing implementations) was the decision taken to change the
> specification for close() , instead of just adding a new posix_close()
> that addresses the problem?
I can't speak for everybody else's motivations -- check the (long!)
rest of the issue tracker or mailing list threads for that -- but I
don't think leaving close() ambiguous was a viable solution. The vast
majority of application code is not going to change to use
posix_close, so we'd still be left with double-close or fd-leak bugs
all over the place. The ambiguity was a serious bug that rendered one
of the most commonly used functions unsafe in a subtle but serious
way.
> 3. [An observation, not a question] As far as I can tell, the Linux
> behavior for close() EINTR semantics is not an isolated case. One can
> see this in the FreeBSD close(2) manual page [1] and in the AIX [2]
> manual page. It seems also to be the norm for other implementations
> (see my next point). HP-UX claims [6] in the documentation to leave
> the file open on EINTR, but unfortunately, the source code isn't
> publicly available to verify this
I'm pretty sure the documentation is correct, though I haven't
verified it myself; IIRC people coming from the HPUX side were adamant
that their choice was correct.
> 4. The interpretation for bug 529 doesn't address a wider issue. On
> several implementations, the FD is always closed *for any error* that
> close() may return.
It fully does address this. You missed the following in the accepted
text for issue 529:
For all other error situations (except for [EBADF] where fildes
was invalid), fildes shall be closed.
Previously the state for other errors was also unspecified.
> That is what Linux does (the FD is released very
> early in the "close" processing, and errors may be reported
> afterward). It's also what FreeBSD does, as documented in its close(2)
> man page:
>
> In case of any error except EBADF, the supplied file descriptor
> is deallocated and therefore is no longer valid.
>
> (By the way, that text seems to have been added in FreeBSD 9.1, in
> early 2012, I presume to document existing behavior after someone read
> this Austin bug report.)
>
> For info, FreeBSD documents the following errors for close(): EBADF,
> EINTR, ENOSPC, and ECONNRESET.
>
> Looking at some historical source code (mostly from [3]) suggests that
> the "close() always closes regardless of error return" behavior has a
> long history, predating even POSIX.1-1990.
I agree with all of this, the whole principle that close should
"always succeed" -- if not you get in a situation of having stuck file
descriptors you can't close, and thus error paths that cannot make
forward progress. However the conclusion I draw from this is that
EINTR is simply an inappropriate error for close. EINTR should only
happen when the syscall was interrupted _before_ it could perform the
main side effect, which in the case of close is freeing up the fd
slot. I understand that, from some people's perspective, the main side
effect of close may be cleanly completing the close operation on the
underlying file/open-file-description (possibly a device node) and any
actions associated with it (like rewinding a tape), but from my
perspective those are secondary to the core requirement that the
application be left in a consistent, predictable state. If an
implementation really wants to allow that degree of control
(determining when the actions on the underlying device have completed)
then it needs to keep an fd handle available to the application so
that the application can retry close.
For what it's worth, having to retry close on EINTR when you don't
care about the device state is a big problem, as it can also block
forward progress for unbounded time. From my perspective, this is the
reason posix_close is added: to allow applications, on implementations
where close produces EINTR and leaves the fd open, to request
asynchronous closing with EINPROGRESS instead.
> For example, in SVR4 for x86 (from the file sysvr4.tar.bz2 at [3]), we
> see the following:
>
> ===
> int
> close(uap, rvp)
> register struct closea *uap;
> rval_t *rvp;
> {
> file_t *fp;
> register int error;
>
> if (error = getf(uap->fdes, &fp))
> return error;
> error = closef(fp);
> setf(uap->fdes, NULLFP);
> return error;
> }
> ===
>
> In the above, getf() can return EBADF. The other errors are returned
> by closef(), but the file descriptor is deallocated regardless of
> errors by setf().
>
> A similar pattern seems to have been preserved into at least late
> OpenSolaris days (verified from looking at the initial commit of the
> illumos source code). There we find the following in closeandsetf()
> (called by close())
>
> error = closef(fp);
>
> setf(fd, newfp);
>
> return (error);
>
> Looking at the code of closef() in AIX 4.1.3 suggests that, as on on
> Linux and FreeBSD, the open file is always released, regardless of
> errors.
>
> For Irix, 6.5.5, I'm not sure (the code is not so easy to quickly
> read); it may be that it does return errors while leaving the FD open.
>
> So, my summary here is that many (perhaps most?) implementations are
> similar to Linux and FreeBSD, but this isn't currently addressed in
> POSIX, so far as I can tell. Should it be?
Again, I agree with the principle; I just think (and POSIX seems to
agree) that EINTR does not belong in that system.
Rich
