Re: core statement on fexecve, O_EXEC, and O_SEARCH
On Wed, 5 Dec 2012 20:42:32 + David Holland dholland-t...@netbsd.org wrote: That may be, but it's still true of file descriptors. Traditionally they're capabilities, and I really don't like the idea of rearranging that arbitrarily and inconsistently. I think to do this correctly, exec and search on file handles needs to be handled the same as read and write: you must open with the corresponding O_ flag, and that then gives you a capability that remains valid until closed (or revoke(2)'d). Ah yes! Modelling this in terms of capabilities is what I think has been missing from this discussion. Or rather, nobody (myself included) seems to have picked up on your previous mention. Of course, possessing a capability, and possessing the capability to grant that capability[1] are two different things. Is it really a good idea that any process that can execute a file should automatically be able to allow another process to do so? Conversely, there's not much point in being able to pass the capability to another process if it needs to already have execute permission. Because passing capabilities around is potentially dangerous, if we're going to add new capabilities we need to either be certain that no restrictions are needed, or add a mechanism to restrict capability passing. Since we clearly aren't certain (if anything, we're certain that restrictions are needed), someone needs to come up with a mechanism. Are you saying what I just said, or something else? I think it's a good idea to know how this thing can be abused before devising a mechanism for restriction. It's ok to say “we're not certain” but taking a guess risks restricting the wrong thing and leaving a hole wide open. I have a fond memory of rooting[2] a friend's (Linux) box using rmt, which his distribution had installed suid root. All his carefully set permissions on devices were rendered pointless by this one oversight. There is a good bit of prior art we can take from in existing or historical capability systems, so designing a suitable mechanism and interface will probably take a bit of reading but shouldn't be very hard. Just remember that the process root directory is also a form of capability, and it shouldn't be that hard to figure out how to make everything interact. This does mean we aren't going to be POSIX-compliant on O_EXEC and O_SEARCH (are there any other O_ flags involved?) but I think it's become clear that this is inevitable. When we're done we can decide if O_EXEC and O_SEARCH and the corresponding other bits should disappear or not when _POSIX_SOURCE and friends are defined to request strict POSIX. There appears to be consensus on deviating from the spec. But it seems clear to me that non-compliant behaviour shouldn't be enabled if strict compliance is requested. Principle of least surprise: not implemented is a hard failure mode and easy to handle; not compliant can lead to an apparent success and a debugging nightmare. We also need to decide if the process root dir and process current dir are equivalent to file handles and are thus capabilities (meaning search permission does not need to be checked again at use time) or not. This could go either way and I don't currently have an opinion. My instinct is to say that consistency is key, and therefore considering all of these attributes as capabilities is the way to go. However, this could change the behaviour of some code. One can imagine a process A which has the ability to request that process B (with different credentials) make permission changes on its behalf. What if A asks to be denied search permission to its own cwd? Of course, this is a contrived example. Perhaps it's not a problem. We're also going to need a way to encode capabilities such that they can be passed down from syscall code into vfs and filesystem code. When the only path requiring alternate forms of security check was truncate, it was ok to deal with it on an ad hoc basis; but now there are a lot of places that need to check either permissions or a capability, and I really don't want to be doing it in anything other than a systematic manner, especially in namei. The obvious answer is to pass around a reference rather than the cap itself. All operations on caps then are handled by functions/macros that set/extract values in the global table as required - no code ever touches them without going through the abstraction. I say global because I'm thinking of the case where caps are passed between processes. But there are other ways to solve that problem. Either way, file descriptors are reduced to being a local alias to the relevant capability. There may even be a cunning way to accelerate that lookup. Julian [1] I hate English. I'm talking about “able to do” vs “able to permit”. [2] Like many insomniacs, I get naughtier as the night goes on. -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: Making forced unmounts work
On Thu, 6 Dec 2012 08:23:07 +0100 J. Hannken-Illjes hann...@eis.cs.tu-bs.de wrote: Looks like this thread is dead. No one beside David Holland is interested and David objects. I take back my proposal. Have been reading the discussion. Don't assume that no contribution means no interest! David wants forced unmounts to work even if a thread gets stuck permanently in a vnode operation. I don't see a way to safely reclaim a vnode from a file system when this vnode is in use by another thread. I think you could take some inspiration from Linux here: it has a very handy umount -l which detaches the filesystem from the tree, but defers the rest of the unmount/cleanup until the fs is no longer busy. This can help in situations where you're not trying to physically remove media but just need to reclaim the mount point, and therefore have no immediate need for a umount -f. Because no process can open anything new on the detached fs, if it eventually unwedges itself somehow it won't get rewedged. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: core statement on fexecve, O_EXEC, and O_SEARCH
On Tue, 4 Dec 2012 08:49:17 + (UTC) mlel...@serpens.de (Michael van Elst) wrote: The descriptor is probably already closed on exec before the syscall tries to use it. Nope. That happens later. I was looking through this code yesterday as the topic interests me. The namei lookup happens pretty early on. I haven't solved it, but the problem seems to be one of context - if you try to execve /proc/self you'll also get ENOENT instead of the expected EACCES. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: core statement on fexecve, O_EXEC, and O_SEARCH
On Tue, 4 Dec 2012 15:30:36 + David Holland dholland-t...@netbsd.org wrote: On Tue, Dec 04, 2012 at 01:58:13PM +, Julian Yon wrote: The descriptor is probably already closed on exec before the syscall tries to use it. Nope. That happens later. I was looking through this code yesterday as the topic interests me. The namei lookup happens pretty early on. I haven't solved it, but the problem seems to be one of context - if you try to execve /proc/self you'll also get ENOENT instead of the expected EACCES. That doesn't make much sense... nor does the procfs_lookup code shed any significant amount of light on it. It's weird, isn't it? I've been staring at the code wondering what I'm missing. Regardless of the pros cons of being able to exec a fd, I can't see how this inconsistency is correct behaviour. -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: Problem identified: WAPL/RAIDframe performance problems
On Sun, 2 Dec 2012 04:04:23 + David Holland dholland-t...@netbsd.org wrote: On Sun, Dec 02, 2012 at 03:22:24AM +, Julian Yon wrote: It's not weird, and there is a gain; it's for compatibility with large amounts of deployed code that assumes all devices have 512-byte blocks. If code makes that assumption, how does the reported block size affect that? Lying is illogical. Code either assumes a specific size (and ignores what you tell it), or it believes what it's told. Either way, dishonesty gains nothing. If code just blindly makes that assumption, it's ignoring what's being reported. You appear to have just agreed with me, which makes me wonder what I'm missing, given you continue as though you disagree. I assume there is or was code in Windows (like we used to have code in NetBSD) that would check the sector size and refuse to run if it wasn't 512. IMHO any time you do the same thing as Windows, you're almost certainly doing it wrong. However, we're talking about hardware here, so you have to also consider the possibility that the drive firmware reports 512 because that's what someone coded up back in 1992 and nobody got around to fixing it. If that doesn't count as broken, what does? (Also, gosh, when did 1992 become so long ago?) Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: Problem identified: WAPL/RAIDframe performance problems
On Sat, 1 Dec 2012 23:46:07 + David Holland dholland-t...@netbsd.org wrote: I don't care about the block granularity of the interface. (Unless I suppose it's larger than the atomic write size; but that would be weird.) If it's smaller than the atomic write size that's equally weird. Because that implies that the designers have made the explicit decision to sacrifice performance for no gain. But there is a cost: they had to write firmware code to emulate that block size. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: fexecve, round 2
On Mon, 19 Nov 2012 08:38:11 + David Laight da...@l8s.co.uk wrote: On Mon, Nov 19, 2012 at 08:08:58AM +, Emmanuel Dreyfus wrote: If you have r-x permission, you open with O_RDONLY and you do not need O_SEARCH/O_EXEC. If you have --x permission, you open with O_SEARCH/O_EXEC I think the standard implied that O_EXEC gave you read and execute permissions. So you can't use it to open files that are --x. No, Emmanuel is right: [...] use the O_EXEC flag when opening fd. In this case, the application will not be able to perform a checksum test since it will not be able to read the contents of the file. You can open with --x but (correctly) you can't read from the file. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Sat, 17 Nov 2012 21:45:02 + David Laight da...@l8s.co.uk wrote: On Fri, Nov 16, 2012 at 12:52:30PM +, Julian Yon wrote: What does this gain over passing a filename around? (NB. I'm not claiming that's an entirely safe model either, but it's already possible). You don't need the executable image inside the chroot. I don't believe that's intended to be possible, and if it is, I'm not sure it's a gain. -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: fexecve, round 2
On Sun, 18 Nov 2012 18:16:00 + David Holland dholland-t...@netbsd.org wrote: On Sat, Nov 17, 2012 at 06:42:50PM -0500, Thor Lancelot Simon wrote: Further, requiring O_EXEC would seem to directly contravene the standard's language about fexecve()'s behavior. The standard is clearly wrong on a number of points and doesn't match the historical design and behavior of Unix. Let's either implement something correct, or not implement it at all. What if the only process that was allowed to fexecve was the one which opened the fd (or possibly extend this to children, but I'm not sure if that's safe), and any other failed with EBADF? Seems to me this would allow the intended usage (tenuous as the rationale is) while closing the chroot based holes that have been discussed. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Sun, 18 Nov 2012 14:31:29 -0500 Thor Lancelot Simon t...@panix.com wrote: On Sun, Nov 18, 2012 at 07:27:27PM +, Julian Yon wrote: On Sat, 17 Nov 2012 21:45:02 + David Laight da...@l8s.co.uk wrote: You don't need the executable image inside the chroot. I don't believe that's intended to be possible, and if it is, I'm not sure it's a gain. I actually think it might be, if it didn't run the risk of blowing up code that wasn't written to expect it. As I've clearly missed it, which text in the spec suggests that? I don't see chroot mentioned anywhere. If we're going to commit this syscall at all, I think it should be accompanied by a new socket option for unix domain sockets, which defaults to off, but if explicitly set to on, allows file descriptors passed across the socket to be used for exec. Or just flag all descriptors passed over sockets as non-executable, i.e. implement the call but prevent that particular pattern. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Sun, 18 Nov 2012 14:08:05 -0800 Matt Thomas m...@3am-software.com wrote: Given that, you can't open a file for read exec, just either read or just exec. So how would verification work again? Seems to me you need a fcntl that would allow you to change the access mode of the file descriptor. Incredibly, the spec defines the illogical behaviour: “Since execute permission is checked by fexecve(), the file description fd need not have been opened with the O_EXEC flag. However, if the file to be executed denies read and write permission for the process preparing to do the exec, the only way to provide the fd to fexecve() will be to use the O_EXEC flag when opening fd. In this case, the application will not be able to perform a checksum test since it will not be able to read the contents of the file.” i.e. If you want to read and exec, you open with O_RDONLY. If you don't have read rights you can open with O_EXEC instead, and you can't read the file you just opened; it merely provides a mechanism to pointlessly use fexecve. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Fri, 16 Nov 2012 08:43:20 + David Laight da...@l8s.co.uk wrote: On Thu, Nov 15, 2012 at 04:02:50PM -0500, Thor Lancelot Simon wrote: Look at that rationale carefully and I think you will see the race condition that it does not eliminate. Talk about a solution looking for a problem! You could create a temporary file, unlink it, copy the executable into the new file, verify the the contents, and then exec the unlinked temporary file. What you've done here is increase the complexity of the attack (win two races instead of one) but not eliminate it. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Fri, 16 Nov 2012 08:34:29 + David Laight da...@l8s.co.uk wrote: On Thu, Nov 15, 2012 at 10:14:18PM +0100, Joerg Sonnenberger wrote: Frankly, I still don't see the point why something would want to use it. How about running a staticly linked executable inside a chroot without needed the executable itself to do the chroot. What does this gain over passing a filename around? (NB. I'm not claiming that's an entirely safe model either, but it's already possible). Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Fri, 16 Nov 2012 20:08:45 +0100 m...@netbsd.org (Emmanuel Dreyfus) wrote: Emmanuel Dreyfus m...@netbsd.org wrote: The spec only requires that the file only needs to be open for reading. That is not completely clear to me. open(2) and openat(2) get a O_EXEC flag, but I did not found other reference of it: fexecve(2) specification does explicitely require a fd open with O_EXEC. Wait, it actually says something: http://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html --- cut here--- Since execute permission is checked by fexecve(), the file description fd need not have been opened with the O_EXEC flag. However, if the file to be executed denies read and write permission for the process preparing to do the exec, the only way to provide the fd to fexecve() will be to use the O_EXEC flag when opening fd. In this case, the application will not be able to perform a checksum test since it will not be able to read the contents of the file. Note that when a file descriptor is opened with O_RDONLY, O_RDWR, or O_WRONLY mode, the file descriptor can be used to read, read and write, or write the file, respectively, even if the mode of the file changes after the file was opened. Using the O_EXEC open mode is different; fexecve() will ignore the mode that was used when the file descriptor was opened and the exec will fail if the mode of the file associated with fd does not grant execute permission to the calling process at the time fexecve() is called. --- cut here--- Let me interpret that for you: (a) If you have read access to the file, you may open the file for reading, perform whatever operation you were doing (e.g. checksum) then pass it to fexecve. *At this point* execute permission will be checked, and if the calling process has such permission, the file will be executed. i.e. nothing says you have to have execute permission when you first open the file. (b) If you don't have read access, but do have execute permission, you can open the file with O_EXEC; this won't allow you to read the contents but will give you a handle for fexecve. As with case (a), execute permission will be checked at this point for the calling process, even though it was already checked when you opened the file. I'm struggling to think of a sane reason why case (b) would ever apply, but in any case unless I'm missing something, NetBSD doesn't define O_EXEC anyway (nor does Linux). Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Fri, 16 Nov 2012 15:46:00 -0500 (EST) Mouse mo...@rodents-montreal.org wrote: Nobody has quite come right out and said what the race supposedly remaining with fexecve is, or at least not that I've seen; the only one I've been able to think of is a TOCTOU race with someone overwriting the file between check and execute, and that one fexecve does not solve. Yes, sorry, should have been explicit. That's the ridiculously obvious one. (It could be tweaked to solve it, by (for example) making O_EXEC (a) required for fexecve and (b) lock the file against writes in the same way executing it does, the way that's behind ETXTBSY.) Which would be contrary to the spec anyway, no? If an implementation needs to be non-compliant to be safe, is that better or worse than just not having it at all? O_EXEC seems to exist on FreeBSD but not on OpenBSD or Linux so I doubt there's much code out there that would work unmodified with such an implementation. It's true fexecve doesn't solve the latter, but the bar _is_ higher; assuming checking involves checking ownership as well as contents, exploiting it requires the ability to overwrite a file owned by whatever user the check is checking for. To make this statement, presumably you're also tacitly assuming checking permissions? Doesn't matter who owns it if it's world writeable. (If, as is probably the case in many such uses, that user is root, I have trouble seeing _any_ issue here - anyone who can overwrite root-owned files pretty much pwnz0rz the system already.) Depends whether they can overwrite all root-owned files, or just specific ones (due to some other exploit). Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Thu, 15 Nov 2012 11:03:15 -0500 Thor Lancelot Simon t...@panix.com wrote: This strikes me as profoundly dangerous. Among other things, it means you can't allow any program running in a chroot to receive unix-domain messages any more since they might get passed a file descriptor to code they should not be able to execute. If there is not some explanation I am missing for why this doesn't basically blow up chroot's security in the very common case where chroot is used to build a W^X environment, I am strongly opposed to ever including this system call in NetBSD. I'm sure the Linux crowd don't care, since they deliberately and proudly punted on actually being able to contain any misbehaving process within a chroot long ago. But that's not us. As you mention Linux, their man page for fexecve states “The file descriptor fd must be opened read-only, and the caller must have permission to execute the file that it refers to.” It sounds like that might be what you're missing? I certainly interpret it to mean that if you can't execve the file, you can't fexecve a FD to it either. From the spec: “The purpose of the fexecve() function is to enable executing a file which has been verified to be the intended file. It is possible to actively check the file by reading from the file descriptor and be sure that the file is not exchanged for another between the reading and the execution.” ...which seems a reasonable enough thing to want to do. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: [PATCH] fexecve
On Thu, 15 Nov 2012 16:02:50 -0500 Thor Lancelot Simon t...@panix.com wrote: Look at that rationale carefully and I think you will see the race condition that it does not eliminate. Talk about a solution looking for a problem! Indeed I do. Fair point. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: suenv
On Tue, 23 Oct 2012 10:32:18 -0400 Thor Lancelot Simon t...@panix.com wrote: On Tue, Oct 23, 2012 at 04:31:52PM +0200, Emmanuel Dreyfus wrote: Background: libpthread is tagged as not loadable by dlopen() in NetBSD-6.0. This breaks PAM modules that are linked with -lpthread or that dlopen() other objects linked with -lpthread. Don't do that, then. You appear to be ignoring the existence of the Real World. You may write all the software you use yourself, but the majority of us do not. If we did, this conversation wouldn't be happening. Emmanuel is making a proposal for a mechanism which will allow him to work around a Real World problem. I am not convinced his solution is the right one, but that doesn't make his problem disappear. Besides, a process wanting to use pthreads is not, itself, a bug. That there is no way for that to happen in this case arguably is. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: suenv
On Tue, 23 Oct 2012 12:16:28 -0400 Thor Lancelot Simon t...@panix.com wrote: On Tue, Oct 23, 2012 at 10:46:53AM -0500, Eric Haszlakiewicz wrote: How did those modules work pre-6.0? I thought pthreads wouldn't work at all when loaded through dlopen because libc would have initialized in non-threaded mode at process startup. [...] NetBSD 6 was fixed to loudly rather than silently fail in such cases. That was the right decision, rather tha allowing application authors to silently hose applicatio users. Yes. There are cases where desperate, ugly hacks to make other people's incorrect code work most of the time are justifiable. The system authentication path is not one of them! I thought he was advocating a more generic kludge mechanic rather than one specific to authentication? Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: suenv
On Tue, 23 Oct 2012 12:21:42 -0400 Thor Lancelot Simon t...@panix.com wrote: You appear to be ignoring the relevant standards. A process is either threaded or it is not, and thus a shared object which may be loaded into arbitrary processes must not use threads. I'm not ignoring the standards, I'm just not ignoring the situation, either. Something which used to soft-fail now hard-fails (correctly), and this has bitten someone who is now trying to work out what would make a temporary fix possible in similar situations. Doing so in authentication software is just insane. In the real world I live in, one needs to be particularly careful with security software, not the other way around. Agreed. Nasty hacks like subverting the protection against LD_PRELOAD on setuid executables are not called for in a case like this. If we resort to them, why should our users trust us to deliver quality software? If you want the wild west, you can find Debian's openssl patches over there -. I'm not advocating his hack, merely noting that there's a Real World reason why it has been suggested. It reminds me of the old joke, “Doctor, it hurts when I do this.” “Don't do that then.” As someone who uses Linux as well as BSD I see exactly the same thing happen “over there”. Sometimes the crazy hacks that distributions put in place are a response to upstream refusing to come to some interim compromise while the real problem gets fixed (if it gets fixed, of course). I agree that this is Not NetBSD's Problem, but I wonder how many people devise their own insane “solutions” to this sort of thing and are put at risk by the lack of an official workaround? I'm thinking particularly of less experienced folk, here. -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
Re: suenv
On Tue, 23 Oct 2012 18:30:21 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: Thor Lancelot Simon t...@panix.com wrote: Nasty hacks like subverting the protection against LD_PRELOAD on setuid executables are not called for in a case like this. Why is it a hack? I immagine it only configurable by root, and I understand it would cleany enable the program to use threads. Or do I miss a point? From Thor's response I suspect that the LD_PRELOAD restriction that I asked you about (that I know is in Linux but wasn't sure about NetBSD) probably is in place. And it's a hack because the real problem in your specific case lies in the design of the module, not in the threads restriction. Julian -- 3072D/F3A66B3A Julian Yon (2012 General Use) pgp.2...@jry.me signature.asc Description: PGP signature
