ban...@openmailbox.org dixit: > Hello, I wanted to ask a few questions about the security of mksh. We are > currently debating its use over GNU bash in our anonymity-centric distro > Whonix, in a very security sensitive package. It appears the closest thing to
That’s nice. Thank you for asking. I strongly believe that mksh is the correct choice for secure shell programming. Thanks for coming towards us actively, in a sane medium (eMail, and possibly IRC, are much nicer than a webforum). > the OpenBSD implementation of ksh thats packaged in Debian. Actually, that package was removed some time ago and is now superseded by mksh. I am also the Debian maintainer of it. And even before that, the pdksh package in Debian was not OpenBSD ksh, but contained a few of its patches (but not the cleanup). > 1. How close is mksh to OpenBSD's oksh code? Are they the same package? OpenBSD’s ksh was used as starting point, and I track commits there, see whether they still apply and improve things, and, if so, take them over. If not, I merely record the time of the last sync. We have all the security-related goodies of OpenBSD, plus lots of other bugfixes and functionality improvements. > 2. Have you or the OpenBSD project performed a security audit of mksh? I have no idea about the OpenBSD project. They would prefer to not hear from either their upstreams or downstreams, anyway. I have been continuously checking mksh code and operation for years, but not with a focus on security-specific issues, but in a more general “bug” scope (which would include, but not limit to, security-specific bugs). mksh is shipped with Android. This prompted a security review by Google’s Geremy Condra, who was “impressed” with the code. As a result, I merely added _even_ more explicit checking code to malloc size calculations. You may want to contact Geremy (see the AOSP Gerrit interface for details) on whatever he has to share about mksh. As mksh was integrated into other projects, such as MidnightBSD, Beastiebox, etc. people were generally positively impressed by the quality of the code. I’ve not heard of anyone who had a bad impression from mksh, ever. > 3. How do both ksh implementations compare from a code correctness point of > view? OpenBSD ksh is pretty much frozen, and is barely more correct than the last pdksh release from 1999. In contrast, mksh is *much* more POSIX-compatible, although, to achieve the best POSIX compliance you can, I suggest compiling *two* versions of mksh: /bin/mksh which can be used as users’ login shell and has all modern functionality, and /bin/lksh which is built with CPPFLAGS=-DMKSH_BINSHPOSIX and the -l option to Build.sh (the Debian package already does this, plus a few more things so that one can use lksh as /bin/sh on an unmodified Debian system). Recently, a bug came to my attention that I was not able to fix right away. As the bug was caused by a feature extension, that feature was removed until such time as we can fix the bug. I’m trying very hard to make mksh the best shell there is, within the scope (e.g. no floating point like ksh93, and no fancy and programmable completioin like GNU bash and zsh, but also no useless bloat, and compact and secure code). I’m generally talking with developers of other shells (such as the dash and posh maintainers, and Jilles Tjoelker from FreeBSD who maintains their /bin/sh, and sometimes David Korn (AT&T ksh) and Chet Ramey (GNU bash)) and users, as well as the Austin Group (POSIX standards committee). While we sometimes disagree, we usually agree to disagree, and work together (unlike OpenBSD, which has an extremely closed development group; it’s very rare that someone can collaborate with OpenBSD developers on something, although I did get the one or other thing across in both directions sometimes). > 4. In general do you audit your base system and its utilities? As time permits, yes. Thanks to the work already done by OpenBSD, there is not as much to do as with others; for example, the secure string functions are used all over the place, and we build with -Wformat and -Werror globally. > I am trying to confirm the facts I've researched and posted in this discussion > thread. > > https://www.whonix.org/forum/index.php/topic,444.msg3598.html#msg3598 MirBSD backdates to the 29th of August, 2002 ;-) means about a dozen years, not just a decade. I was an OpenBSD user, and before that, created a GNU/Linux distribution. mksh indeed supports a lot of GNU bash extensions which OpenBSD ksh doesn’t. Most are shared by AT&T ksh93. I’m willing to help answer migration questions, either here or in the #!/bin/mksh IRC channel on freenode (yes, that’s really the channel’s name), including the differences between “full” mksh and “legacy” mode (for better POSIX compliance). oksh is not a drastically improved version. The code was cleaned up, including removal of portability code, but only a handful of bugs were fixed, and two or three new introduced. (Actually, "oksh" is OpenBSD’s /bin/ksh plus glue to run it under GNU/Linux, DeliLinux to be specific. Its first version featured code stolen from mksh without adhering to mksh’s (very liberal) licence; that was subsequently removed, but oksh from DeliLinux is distributed under (restrictive) GNU GPL.) > https://www.whonix.org/forum/index.php/topic,444.msg3615.html#msg3615 OpenBSD doesn’t say about the frequency of their audits. I know that their modus operandi is, in general, “if we see a bug, we look through the entire base system for similar occurrences of the bug”. In MirBSD, we do more or less the same, modulo manpower. But, as a fork of OpenBSD, we build upon their wonderful work. OpenBSD base includes their ksh, but, as I said above, it’s more or less static. It fulfils their job, and that’s it. On the other hand, I personally want to code in shell (as opposed to PHP, Python, Perl or even C) as much as possible, and I have a very personal drive to make mksh the best shell to do that there is, which is why we have active development. mksh and OpenBSD ksh indeed do share “some” code. At some point a decade ago, that was about 100%, but as I introduce bugfixes and new code, and rewrote the parser to be properly recursive, etc. the percentage gets less. | - Even if ksh was audited and even if mksh and ksh are | compatible (?), no one claimed that mksh was audited. We have an independent security review from Google, see above. | Also mksh is much less popular. Got an older by_vote popcon statistic | by Debian on my hdd. This is because the "bash" package is Essential, which means it exists on virtually all Debian systems, while mksh isn’t. | So I would assume that mksh attracts much less attention by | potential people who could be interested to look for | vulnerabilities. The installation base of mksh is more than a billion, mostly thanks to Google’s Android. It is (with probability approaching certainty) the Unix shell most widespread in use. It’s also extremely portable ☺ I’d say that this refutes this point. | 2. From an auditing perspective we are only interested in the mksh code And indeed, mksh is a core project, and has received much more attention from the MirBSD team and contributors than the rest of the base system. This means it’s probably the part of our CVS repo that’s in the best quality. | I can see there is huge confusion because of the semantics here. A | short history of ksh: ksh was a proprietary shell in one of the | ancient Unixes. At the turn of the millennium it was released under | the name: pdksh in the public domain. OpenBSD took it over and then | it audited six ways from Sunday, cleaned the code up and modernized | it then called it oksh, MirBSD another security oriented fork took | that exact same audited code and further slimmed it down, sharing | any code fixes back with OpenBSD and called it mksh. Allow me to fix this: AT&T ksh88 was the proprietary shell, yes. But pdksh was an independent development, based on the Public Domain Bourne Shell (clone) by Charles Forsyth (Vitanuova). In the meanwhile, AT&T ksh93 emerged. pdksh deve- lopment stopped in 1999, but it had been imported into many operating systems (the BSDs, QNX, Plan 9’s APE, Debian posh, etc). At some point, ksh93 was released under more and more liberal licences, even OSI- approved ones, but it’s still not free enough for the BSDs, and pretty large and very complex code, not easy to understand even for me. OpenBSD did not take over pdksh development. They just improved the quality of their base system, which happened to include, under just the name "ksh", pdksh. They also threw away *all* compatibility code. In MirBSD, we inherited that; I liked their cleanup and decided to start working on the shell as well. I eventually brought not back but new compatibility code, done in a more clean matter than before. But I also slimmed it down and fixed tons of bugs, and wrote new code. OpenBSD do not, in general, wish to receive code from me. Just "ksh" can refer to any of these, or even MKS ksh (the one famous for David "Korn Shell" Korn to ask "why does your shell suck so much?") or even more others. I’ll have to read through all of that thread, but please forward my answer, e.g. from http://news.gmane.org/gmane.os.miros.mksh (you get article links), there first, to have fodder for new discussion. I’m catching up on work after an extremely busy (otherwise) time, right now, both privately and on $dayjob. Actually, since you sent this to miros-discuss@ originally, not to miros-mksh@ (which only exists for a year or so), this thread is at: http://thread.gmane.org/gmane.os.miros.general/11016 bye, //mirabilos PS: Frank (from my signature) is a zsh developer/committer… -- FWIW, I'm quite impressed with mksh interactively. I thought it was much *much* more bare bones. But it turns out it beats the living hell out of ksh93 in that respect. I'd even consider it for my daily use if I hadn't wasted half my life on my zsh setup. :-) -- Frank Terbeck in #!/bin/mksh
