[linuxkernelnewbies] Review of Operating Systems

Wed, 03 Dec 2008 18:31:39 -0800

http://tunes.org/Review/OSes.html

Review of Operating Systems

[ FAQ | Sitemap | Search | Collaboration
Wiki | Subprojects | Review | Mirrors ]

This page has been superceded by the project Wiki OS Review area.

Original Free OS Projects

  • 2K (formerly known as SPINE) is a distributed, reflective, component-based, adaptable operating system being built by researchers at the Systems Software Research Group from the University of Illinois at Urbana-Champaign and the Systems and Communications Group from the Universidad Carlos III - Madrid.
  • Aegis (developed at MIT's Parallel and Distributed OSes group) is an OS based upon the idea of an "ExoKernel" (much like the NoKernel idea behind Tunes: there is no more runtime kernel in the OS, which yields up to 10000% performance gain). Also see D. Engler's page, and the various Exokernel and networking papers.
  • Olin Shivers' Express project at MIT developed ML/OS, an OS in ML, by combining SML/NJ with the Flux OS Kit, reusing the web server from the Fox project.
  • The Flux project at university of Utah has been developing an OS toolkit (mirrored in Australia), now release 0.97, so anyone can make their own OS from bricks. It is also developing its own Fluke microkernel on top.
  • HOPE at the Oregon Institute of Science and Technology, a set of primitives on top of a virtual machine, for optimistic programming, i.e. guessing the result of operations in order to increase parallelism.
  • Anders Gavare's Yoctix is an emerging operating system very much inspired by BSD Unix, but implemented fully from scratch and following its own path. Anders' goal is to educate himself and to come up with a full documentation, written as he progresses, of how to write such a thing. After only 6 months of work, Yoctix boots /sbin/init. Even cutting corners as he did, this is very impressive. Definitely worth a look.

    Free OS projects in early stages of development

  • "Joker" Josh MacDonald's OSDEV
  • OS-DeViLs project (with an hardware independence page here), is a federation of people hacking new OSes, so they can debate ideas,
  • Michael Gerhards' UniqueOS plans to make an OS based on no fixed data structures, using a technique called tag items, borrowed from AmigaOS. More information and documentation can be found on the site. A few sources can be downloaded.
  • Jascha Wetzel's XOS aims at being an object-oriented, distributed, preemptively multitasked/threaded, message-passing based system. While this would sound like pretty standard marketroid talk, the site contains very interesting, detailed and clear documentation about what the author has in mind, an object-oriented microkernel where every part of the system does one thing and does it well, without any of the other parts knowing the details; a definite plus for scalability and future expansion. The author also cares to define the concepts he uses, so have a look at it even if you don't do C++ in your sleep. The actual implementation hasn't started yet, but definitely a spot to watch.

    Apparently dead OS projects

    The following OS projects seem to be dead, as in not having been upgraded for years. However, they are still worth mentioning, as learning tools, and also in the hope that maybe someone will want to pick up the work where it stopped.
    • Kinfira's Antares OS for x86 PCs, that seems pretty much dependent on x86. Intended to become a microkernel type of design; left dead at function loader and draft filesystem stage.
    • EOS (ask Cleo Saulnier) was an operating system specially designed to run games under. The project looks like it has been abandoned. If you're interested in such a project, probably the best would be to mail Cleo and see if you can profit from what has already been done.
    • LOS was meant to be another OS for gamers, like EOS. However, this one booted and worked, and there were a few games for it (like an arkanoid clone).
    • SQOS is a completely conventional OS that was being written entirely in assembler by two teenagers as a programming challenge (much like KOS or LittleOS). They seem to have got fed up with it though.;
    • Bart Sekura's TINOS aimed at making a small, multipurpose OS that could be used as a learning tool for the author and anyone else that cared to use it. However it has been last updated in march of 1998.

    Educational OSes

    These are instructional OSes developed and used in some Universities for their OS courses. They are freely available, and have some docs, too. In portable C, unless stated otherwise.
    • NachOS from Berkeley, demonstrates the principles of the traditional unix-like model.
    • Xinu, a widely respected "vehicle for teaching Operating System design concepts". It supports most modern OS functionality, including message passing, tcp/ip, and an independent shell.

    Popular Commercial OSes and their clones

    Original contributions from Commercial systems

    • IBM's OS/2 Warp is one branch of what was originally a joint development between Microsoft and IBM, the other branch being known today as Microsoft Windows NT. It can run most DOS and Win 3.x programs seamlessly, as well as some Win9x software.
      While not abandoned by IBM, it is not very actively marketed by them anymore. However, a company has licensed the right to market an up-to-date OEM version of OS/2: eComStation. Also, OS/2 has long been actively supported by a legion of developer-users and enthusiasts, and good development information is available on the Team OS/2 and Seattle OS/2 users group sites.
      FTP sites for OS/2 software: Hobbes, FTP-OS/2
    • Tao OS (now called Elate) is the only OS that currently can distribute code on heterogeneous underlying networks (here their internet contact, and an old introduction to Tao OS). Was renamed Tao OS from TAOS because of legal problems.
    • The Taligent company is the result of some counter-nature cross between Apple and IBM ;-) it worked on a brand new operating system, the CommonPoint application system, that attempted to change the basic programming paradigm, and which has been released on top of AIX in summer '95. To know more about it, you can browse the first two chapters of the book Inside Taligent Technology by Sean Cotter. All in all, it looks like yet another failure due to C++ introducing unmanageable complexity throughout the system.

    Operating system for embedded devices

    As embedded systems (PDAs, cellphones, point-of-sale devices, VCRs, industrial robot control, or even your toaster) become more complex hardware-wise with every generation, and more features are put into them by the day, applications they run require more and more to run on actual operating system code in order to keep the development time reasonable.

    The good:

  • Nexus' coniX small romable embedded operating system for ARM processors.
  • Sun's JavaOS (with a more technical page at JavaSoft) was a standalone virtual machine not running on top of any other OS; mainly targetted at embedded systems. Proprietary project dropped.
  • Palm Computing's PalmOS for the Palm Pilot and soon a flurry of other mobile products. Currently the leader OS for PDAs, has many applications and supporting companies.
  • Microsoft's attempts at fitting a large unstable desktop-PC operating system, with a hammer, into the stringent requirements of embedded systems are here: Windows CE (which is about to lose its 'CE' part so as to confuse people a bit more) and their latest monster, Windows NT Embedded 4.0.

    Commercial Unices and beyond

    Lots of commercial vendors base their system on the Unix family of design, as standardized in POSIX.
    The existence of free Unix systems like the great Linux forces them to find justifications for charging so much for systems that were so bad; hence, recently, significant OS research has been done by commercial companies, even though the benefit for users and developers is not obvious, as "protected" research is by definition not beneficial to people.
    • Solaris-MC: Sun is working in making a distributed OS out of Unix.
    • Sun's Spring System (hmm, dead link at the moment).

    DOS-class systems and extensions

    Because DOS has been such a phenomenon in OS history, that it ought to have a place here, despite its absolute nullity (the only service of it that is used rather than worked around is the filesystem, which is the worst piece of junk ever implemented).
    • A well-known company in Seattle has produced and mismaintained the most crappy OS ever, known as DOS. The dreaded name need not be mentioned here.
    • Of course, there exist lots of DOS emulators for lots of non-DOS systems.
      For instance, Linux has quite a good free DOS emulator running on i386 architectures (also runs under *BSD). But it is actually a hardware emulator, and requires that you use an actual DOS software (e.g. OpenDOS or FreeDOS) to run on top of the virtualized hardware. Newer versions of DOSEMU will be released as a bundle with FreeDOS.
    • Other commercial DOS clones include:

    Losedoze-class systems

    The same company consistently produces the worst wimpy OS, hence this section...
    • The The Cygnus GNU-Win32 library allows you to simulate a GNU system on top of Win32, thanks to the cygwin32.dll
    • Here are links to the unnamed horrors: *NT Workstation and *NT Server; *95.
    • WINE free WINdows Emulator for Unices (also here).
    • Jason Filby's ReactOS project, aiming to produce a free Windows NT 4.0 clone distributed under the GPL license. The developers of the project keep a low profile, however they are making big strides. ReactOS is now able to boot and start simple win32 console-based applications. Best luck to them for the rest of the immense work.
    • Note that there also exist commercial clones, among which the Sun WABI that runs 16-bit Windows apps (soon 32-bit apps too, they say) on your X/Unix workstation. Also exists for Linux, as commercialized by Caldera.

    OS Related Pointers

    • Indices about OSes and related subjects:
      • The FAQ from the comp.os.research newsgroup; also its archive; it is a good FAQ, and anyone interested in OS implementation should have read it.
      • 500mhz.net's operating systems index, sorted by date of discovery (not updated in a while).
      • Brad Appleton's links to operating systems news sites and development resources.
      • Sven Paas' comprehensive german page (in English). Now moving here ?
      • Patrick Bridges' arizonian page
      • Christopher Browne's Novel OS work, featuring a detailed description of several operating system projects.
      • The OS Faq site has a lot of interesting pointers to operating system projects, and answers a lot of questions on which OS programmers usually bump, covering topics such as filesystems, GUIs, device drivers, and existing OSes such as Linux or Windows NT. Thanks to Cory Rauch for the link.
      • Vinny Cahill's [MIA] irish page
      • Hyogun Lee's own OS page.
      • Kang Sung's OS page in korea.
      • OGI's oregonian page
      • Nemo's OSes links
      • Embedded and Real-Time OSes.
      • Patrick Gunn's OS page and webring.
      • Seoul national university OS labs operating systems links
      • SIGOPS Create your own OS tutorial, describing the basic steps involved in writing an operating system, if you don't know where to start.
      • XOR coders OS development pages and tutorials. The information there isn't very deep yet, but might be a good source of information for beginning future torvalds.
      • Yahoo's page about Operating Systems
      • John Fine's home page contains excellent information for helping starting developers get over the bootstrap phase on x86 archiecture, by providing numerous well-done explanations and sourcecode about protected and v86 mode, linking kernels, reading them from bootstrap (even directly from a file) etc. Pretty much a must read if you start a project on x86.
      • Gustavo Zamboni's computer architectures and OSs on the web, a very comprehensive list of links, sorted by the platform they relate to. No description is made of what you will find in any of them (besides the site name), but the list is very large and interesting.
      • AltOS, "practical alternatives to microsoft operating systems". A guide of all 'alternative' systems like MacOS, Linux, BeOS, with screenshots and descriptions.
      • OS/A65 is an insane (and functional project): a multitasking operating system for old CBM computers like the C64, with TCP/IP and all. Not useful for much, but still worth mentioning.
      • OS doc at ftp.funet.fi
      • Distributed Algorithms & Systems
      • Process Migration & Load Balancing
      • Checkpointing
      • ANDF, an object format that allows for executables that can be run on any architecture, by translating the code blocks to actual opcodes at load time, rather than at compile time. Structure information is still retained (as opposed to bytecode where this vital data is lost) so the created code still runs at C speed. This excellent format is notably used by the Tendra compiler.

    • Here is some information on memory management issues and common policies:
      • People working on Garbage Collection have a LOT to teach us on memory management.
      • A Virtual memory tutorial, describing the motivations behind that common technique in modern OS'es, its theory and potential problems, and actual implementation issues.
      • Proposal for a Zone allocator for Linux, and the homepage for Linux memory management, by Rik Van Riel. Quite informative about how kernel memory allocation works in an OS used for real-life applications outside of the testing lab, some problems with its current scheme and potential solutions.
      • Dynamic memory allocation implementation in userspace under Linux/ glibc. Different policies than what the kernel does.
      • The BGET memory allocator; another set of policies for userspace memory management and their implementation.
      • FreeBSD's newvm, the new memory manager for that free BSD/Unix clone. Another design and implementation for a real-world system, worth a look for anyone developing a memory manager for their OS.

    • Here are some research laboratories interested in operating systems (send me more addresses):

    • Here is some code that you can link into your OS and relieve you from the hassles of blindly getting it to boot, by providing you boot software and basic I/O:
      • Etherboot and its successor NILO allow x86 PCs to boot an operating system across the network; they include code and tools for producing EPROMS that can be placed on the corresponding network card sockets so that diskless PCs can download and start an operating system kernel, using standard protocols (bootp, dhcp, tftp). They can boot systems like Linux this way, but are easily adaptable to your own.
      • The Flux project's OS toolkit.
        The Flux OS kit is a complete modular infrastructure that provides all the bricks needed to easily build an OS, focusing on what interests you, and reusing components for the rest. Newbies have reportedly transformed Unix programs into standalone OSes in a matter of hours, using it.
      • GRUB the grand unified boot loader for PC and compatibles. Originally written by Erich Boleyn, it has been taken over by the GNU project, as Erich didn't have any more time for it, but GNU depend on it for the HURD. The original (with documentation, and also specification of the multiboot standard) could be found here; the new GNU GRUB is here with even more impressive functionality.
        As mentioned, GRUB is used by the HURD, and it can also boot *BSD, Linux, DOS, and more. It's got facilities for accessing the filesystem at boot time, so you can safely write your loader as just another 32-bit process.
      • Dolphin bootsector is a bootsector primarily designed to boot Dolphin, as the name says, but the author is nice enough to have written documentation about how to boot your own OS project from it; it handles all protected mode and paging setup for you; the whole thing fits in 1 KB so it could be useful when you don't have much room, like on a floppy.
      • SOLO the ShagOS boot loader, includes useful x86 debugging features.
        SOLO not only includes boot-time filesystem access, but extended IO and debugging facilities; might be great to hack your kernel. Its license is now unrestricted, so that you can now freely use it and distribute it with your code. SOLO is definitely worth a look.

    • Here are some sources of inspiration for people developping a 32-bit OS on PC's:
      • DOS extenders in the x2ftp archive (some interesting docs around, too, particularly the PCGPE)
      • Free Embedded systems from Cygnus FTP
      • Some 32-bit FORTH systems from the taygeta archive
      • The Guide to x86 bootstrapping describes the operations performed by a PC compatible computer BIOS at boot time. Useful for writing OS boot loaders, boot managers and such.
      • David Lindauer's software
      • FAQ about how to create bootable CD's for PC in the "El torito" format. Useful when you'll have written an OS large enough to fill a CD..
      • os-dev page for the intel architecture
      • 500mhz.net operating system development resources
      • The Linux Assembly-HOWTO about better (more portable, maintainable, seamlessly integrated) ways to include assembly code in your projects (shameless plug for Fare).
      • Gareth Owen's Gaztek site, which contains a lot of information on OS development and programming the PC hardware, as well as a gimmick OS (GazOS) and even a small webserver (ghttpd).
      • and of course, the code for all the free OSes above, including Tunes.

    • PC hardware information
      Efficiency-aware OS implementors must have some idea of how the underlying hardware behaves.
      The cheapest bang/buck for a complete system currently is available through old or new PC-compatible computers.
      • OS programmers might be interested to get the CPU specs directly from the chip builders: Pentium, Pentium II, and Pentium III manuals, as well as other documentation from Intel, Cyrix, and AMD.
      • Unofficial CPU information Very extensive information (including undocumented features) on the x86 family of processors is available at Robert Collins' famous x86 site and Christian Ludloff's sandpile.
      • A great source of information, and a useful tool for building and testing x86-based OSes is Bochs that fully emulates in software the 386+ PC hardware, on any 32-bit+ machine. Able to boot Linux, FreeBSD, Win95, Roadrunner and many other operating systems.
      • Hardware and low-level programming information on common PC motherboard components is available from Craig Peacock's Interfacing the PC series : parallel port ( standard, enhanced), serial UARTs, keyboard, and the Programmable Interrupt Controller.
      • There is an impressive amount of documentation on PC architecture and associated peripherals (NICs, ATAPI/IDE, VGA, mouse, Flash memory.. and also higher level like all network RFCs) at the Ronis site. A definite good starting point if you need documentation for writing your own system.
      • PC demo coders have also gathered information about the underlying hardware, for instance on cdrom.com's demo coding hardware pages, or Cubic's very extensive hardware archive.

    • Other hardware
      • Another widely used (albeit nowhere as much as x86) is the PowerPC. Motorola provides programming documentation in Adobe PDF format on their tech support site., as well as documentation of their new MMX-alike AltiVec technology. You can find some more programming info (also in PDF) at this place.
      • The MC 68328, also known as the Motorola Dragonball microprocessor, the heart of Palm Pilot PDAs for example, is also of interest as it is a very decent mobile platform for running a modern OS.
      • The CPU Info Center covers all mainstream CPUs in the current competition.
      • MISC technology is not mainstream yet, though very promising, and already the best computation-horse-power/electric-power-consumption ratio. See Jeff Fox's page (also Ultra Technology?) the MISC mailing list (also at Elijah labs) or iTvc's page. (the SISC312 seems around the same design, too). If you're interested in CPU design, Jeff Fox sells a DYOP (Do Your Own Processor) kit based on FPGA...
      • The Chip Directory talks about chips in general, CPU included, but not exclusively
      • For a historical point of view, check this page about Great Microprocessors of the Past and Present

    • Other OS-related pages
      • The Tunes Review page on Virtual Machines
      • Loughborough University high-performance networking and distributed systems archive
      • An Operating System Vade Mecum by Raphael Finkel
      • Dynamic recompilation resources: good introductory papers and concrete applications on dynamic code generation (ie. in reflective sytems) and recompilation (ie. in emulators that need to dynamically translate emulated code to native opcodes, for speed).
      • About UNIX:
        • UNIX Reference Desk
        • An overview of the POSIX standard.
        • A nice article about UNIX literacy
        • Unix Haters are many, who could experience how life used to be better with other systems, and had to use Unix as these systems died commercially. Of course, Unix has made progress since the early 1990's, so not all of the critique still applies; but there is something to be learnt from ancient lore. The Unix Haters Handbook is a book meant to explain the worst of Unix. From what the webpage shows, the book doesn't seem to do much justice to modern Unices, although it shows that what is presented as "great innovation" in the Unix world is but getting on par with decades-old systems.
      • How reliable is your OS? How good is its support? How fast are dangerous bugs fixed? Check the ping o' death page to see...
      • How to write a systems paper (get a text version by sending body "send advice papers" to [EMAIL PROTECTED])

    More netsurfing

    To Do on this page

    This document last modified on Sunday, 29-Oct-2006 13:02:09 PST. See the Changelog

    Webmaster

    Reply via email to