On Sat, Mar 11, 2017 at 12:57:59PM -0700, Theo de Raadt wrote: > > On Sat, 11 Mar 2017, ropers wrote: > > > Was 32-on-64 compatibility somehow easier to achieve on the Linux side? > > > Or did they just keep throwing code and more code at the problem because > > > they felt they really, really had to have this? It's that kind of idle > > > curiosity. If nobody's interested in explaining or hearing this > > > explained, then sorry for the noise. > > > > There's (at least) three parts to such compat for the kernel: > > > > 1) the low-level kernel<->userspace boundary handling. > > A 32-bit process has a different pmap, adding code and requiring the > > uvm layer to indirect or test which is in play for its operations. The > > syscall/trap interface is quite different too; amd64 is sooooo much > > nicer for syscall entry and %fs/%gs handling than i386. You don't > > want that shit in your nice 64bit kernel! > > > > 2) ABI mapping > > So you called some syscall and it has the result to copy out to > > userspace. Oh wait, which ABI is this process using, the 64bit ABI > > that matches the kernel's structure layouts or a completely different > > ABI that requires checking for overflow and then repacking structures > > to match the 32bit ABI? Yay, we get to think about that every time we > > do copyin/copyout! Let's see, 490 calls to those two functions in 119 > > files (not counting sys/arch/*). OSes that support this tend to > > introduce an abstraction layer to reduce the number of those that are > > needed, but that's still cognitive load. > > > > 3) interprocess handling > > a) 64bit -> 32bit > > You weren't really planning on having gdb64 and gdb32, so at least > > ptrace() has to be capable to letting a 64bit process manipulate a > > 32bit process. Go look at FreeBSD's sys/kern/sys_process.c for > > example and examine the COMPAT_FREEBSD32 blocks. They've done it, > > obviously, but it's not simple, beautiful code. A comment like > > > > * This CPP subterfuge is to try and reduce the number of ifdefs in > > * the body of the code. > > > > is a warning that feature is costing you a chunk. > > > > b) 32bit -> 64bit > > Maybe you ban 32bit ptrace() of 64bit processes (most do IIRC), but > > you still want 32bit processes to be able to use sysctl(KERN_PROC) > > and see 64bit processes > > > > > > Implementations have jumped through all those hoops, but at what cost in > > complexity and security holes, and there *have* a been a bunch! A google > > search for "security hole in 32bit compat" immediately turned up this > > article: > > https://lwn.net/Articles/406466/ > > > > Adding "FreeBSD" or "NetBSD" to the search turns up hits for them too. > > This stuff is *hard* and the benefit is...? > > > > Nope, don't want it. > > Thanks Philip. > > By the way the history of OS compat in *BSD is interesting. > > Chris Torek built his LBL sparc code to the SunOS ABI. It was > incomplete. But enough to bring the system up. > > I pulled his sparc code into NetBSD. At the time NetBSD was > transition and even extending the machine-independent system call API > in the not-yet-released 4.4BSD-lite2, rather than having every > architecture contain it's own table. > > Torek's API wasn't good enough to handle structures which needed > modification on the way to/from kernel/userland, so I created the idea > of a zone on the stack where arguments could be tweaked before being > passed on. > > Thus was born the first real compat layer. Compat HPUX was created > very soon, then compat for SCO Unix, some FreeBSD compat, and then > Linux compat which was far nastier because their socket system calls > had been added in a really adhoc fashion. > > There's another problem Philip hasn't brought up. > > 4) The people who rely on the compat layers don't care enough to > maintain it. The people who work on the mainline system don't > care about the compat layers because they don't use them. The > cultures aren't aligned in the same direction. Compat layers > rot very quickly.
Although I don't use compat, this is incredibly interesting information from both Phillip and Theo. Thanks to both of you.
