As many of you know, we have been interested for quite a while in enhancing PXE support in Etherboot. Work by Peter Lister and Vasil Vasilev added prefix code that allows an Etherboot image to be loaded by PXE. This was recently followed by Michael Brown's UNDI driver for Etherboot. We now wish to take another significant step in this direction.
Michael Brown has recently drafted a plan to add significant PXE support to Etherboot. Implementing the plan will require about three months. One of the main results of this effort will be to allow us to generate Etherboot ROMs that are reasonably PXE compliant and which support PXE loader code such as PXELINUX.
One might well ask, "Why are we doing this?"
Let me give some of what I think are good reasons for enhancing Etherboot support for PXE. I'm sure others will have some thoughts as well.
- Etherboot has been moving in the direction of more comprehensive network booting for a long time, with efforts like support of .zlilo, .zpxe, and grub. Support for PXE is yet another effort in the same direction, which gives people a way to load PXE loaders via Etherboot.
- Considering that Etherboot and PXE rely on many of the same underlying network protocols (UDP, DHCP, TFTP), we have a lot of the required functionality already implemented. Additionally, Eric Biederman's work with Etherboot has generalized Etherboot's internal structure to make it easier to support some of the things that are required for more comprehensive PXE compliance. We further believe that we can do a superior job of PXE implementation over proprietary versions because Etherboot's open source nature will allow more people to usefully contribute to the effort.
- We have often joked about "World Domination (and fast!)". Since PXE has become the network booting protocol that manufacturers generally prefer to include in commodity motherboard BIOSes, it seems sensible for us to support it. This also fits in with goals that Ken has mentioned in the past, one of which is to make Etherboot more attractive to hardware vendors and other commercial interests. This, in turn, potentially increases the number of contexts in which Etherboot may be considered as an option for network booting.
- The PXELINUX package has become a popular way to load Linux. H. Peter Anvin, author of PXELINUX, has for some time expressed an interest in free PXE software that would support PXELINUX. By adding the required support to Etherboot, we will be able to support a large body of users who wish to use PXELINUX. With the extensive number of Ethernet cards supported by Etherboot, PXELINUX users' hardware options will be significantly increased.
So, the answer is basically that "we can", and "we think it's a good thing". Giving potential users of network booting more options is valuable, especially since we're looking to implement an existing, accepted standard. Beyond that, we have an opportunity to bring talented people together to be creative and work on an interesting problem. This task certainly qualifies as interesting, and I think it will be a significant advance for the Free Software community. I think everybody wins.
As another side-benefit, by increasing the contexts in which Etherboot can be used, we can demonstrate Etherboot's various other advantages as well. We've got some nice multicast TFTP support that's open source, and we are already working on IPV6 support. Multi-architecture support is another cool and cutting edge benefit we offer. All this and PXE too.
For those who are interested, I have included a summarized version of the development plan. From this plan, you can see that enhancing PXE support in Etherboot will require a substantial amount of work. To allow Michael to devote significant attention to this effort, I have committed to funding him to be primary developer during the projected three months of development time (providing income he'd otherwise have to obtain from taking on unrelated client work). Additionally, I and other developers have cleared time on our schedules during this period to provide testing, debugging, and whatever other support we can to help.
I hope everyone will join me in supporting Michael and this effort through testing, debugging, documenting, and giving our best ideas and suggestions, as we look forward to some very exciting months of productive development, resulting in a significant contribution to the free software community.
With many thanks to everyone who has helped get us this far,
Marty
======= Summary of Plan =======
This is a summary of the plan. For more information on any of the items, please feel free to ask for more detail. The stages are as follows:
STAGE A - Implement high-level PXE API support ("pxelinux")
STAGE B - Implement low-level UNDI support ("EB->PXE")
STAGE C - Implement UNDI & BC loader support ("PXE ROM")
STAGE D - Optional extrasTasks within stages A, B, and C will be generally implemented in sequential order; optional steps in Stage D will be implemented as time permits, after all work in stages A, B, and C is complete.
STAGE A - Implement high-level PXE API support ("pxelinux")
1. PXE image support -- Complete the code currently #ifdeffed under RAW_IMAGE
and marked as incomplete. This enables PXE images to be loaded which can then
be used to test Etherboot's PXE stack.
Milestones: Ability to load and execute an Etherboot .zpxe image. Ability to
start pxelinux. It should fail with an error message indicating no PXE stack
present.
2. !PXE API stub -- Resurrect the FREEBSD_PXEEMU code, add a !PXE structure
alongside the PXENV+ structure, implement a real-mode entry point. Payload for now
is a routine that simply prints out the PXE API call number and returns a
failure code. This allows external programs to make PXE API calls to Etherboot,
so those calls can be tested.
Milestones: Ability to start an Etherboot .zpxe image (with PXELOADER_KEEP_ALL
not defined) and see the appropriate sequence of PXE API calls being made
(the stub prints out the API call numbers). Ability to start pxelinux and see
the appropriate sequence of PXE API calls being made.
3. PXENV+ API stub -- As above for the PXENV+ structure rather than the !PXE
structure (hook Int 1a as a means of locating the PXENV+ structure). This lets
us support both versions of PXE, where the calling convention is the only real
difference.
Milestones: As above. Note that Etherboot's pxeloader has never been tested
successfully with a PXENV+ structure (due to lack of availability of a
working example), so the pxelinux test should be done first.
4. E820 map mangling -- Implement an E820 intercept routine that hides Etherboot
by removing its areas of memory from the BIOS memory map. (Loaded PXE images have
no way to detect that Etherboot is using high memory. A loaded image could
therefore easily trample on Etherboot, thereby destroying the PXE stack.)
Milestones: Ability to load an Etherboot .zpxe image and see that it relocates
itself to a different location (according to the relocation message printed at
startup). Ability to return from loaded .zpxe image to original Etherboot via the
pxeprefix exit path. The "EB->PXE" message should be visible, followed by
the usual blurb from a failed image load.
5. High-level PXE API call implementation -- "High-level" here is defined as
anything that doesn't require changes to the Etherboot driver code. This
equates to the "TFTP API" and "UDP API" from the PXE spec (i.e. excluding the
"Preboot API" and the "UNDI API"). This would not include multicast TFTP support.
These are the easiest API calls to implement and do not involve structural
changes to the rest of Etherboot.
Milestone: Ability to use pxelinux.
6. Silencer -- Ensure that Etherboot is capable of running silently (i.e. without
outputting progress messages to the console) while being used as a PXE
stack. This cosmetic work should be done so that a release can be made.
Milestone: Ability to run pxelinux without any console interference from
Etherboot.
STAGE B - Implement low-level UNDI support ("EB->PXE")
7. Basic Peripheral UNDI API calls for single driver -- Implement all calls
that the Etherboot UNDI driver uses (except for those dealing with transmit
and receive). Initially only the rtl8139 driver will be modified (this card
will be used for initial testing). The loader routines remain stubbed out at
this stage. Implementation of these API calls is required before the UNDI
transmit and receive routines can be tested.
Milestone: Ability to load undi.zpxe and see it detect the rtl8139 NIC exported
via UNDI, complete with I/O address, IRQ number and MAC address. Transmit and
receive will fail; these are not yet implemented.
8. Non-IRQ context transmit and receive for single driver -- Implement
PXENV_UNDI_TRANSMIT and PXENV_UNDI_ISR. Add internal driver API call irq(),
a multi-function call that can be used to enable, disable and/or acknowledge
interrupts at the card level. This interface supports the required functionality
in a generic way *without* adding a large number of extra internal driver
API functions. Goals are (a) avoid breaking the current, polling method of
using the drivers and (b) avoid making the API complex to the point of
discouraging new driver authors.
Only the rtl8139 driver will properly implement this functionality at this
stage; a dummy irq() function will be added to all other drivers so that
they may still compile.
Milestone: Ability to load undi.zpxe and use the rtl8139 NIC exported via UNDI.
9. IRQ context test routine -- Implement an alternative IRQ handler to
trivial_irq_handler that does actually call PXENV_UNDI_ISR. This is relatively
straightforward because we are at that point in real mode and wanting to make
a real-mode API This provides a way to test that our PXE API can be called in
interrupt context and that calls to PXENV_UNDI_ISR in this context will succeed.
Milestone: Ability to use undi.zpxe (booting from a "real" PXE, not
Etherboot-exported PXE), with the alternative interrupt handler instead of
trivial_irq_handler, to download and execute an image.
10. IRQ context calls -- Ensure that our real-mode PXE entry point can be
called in IRQ context and can successfully handle PXENV_UNDI_ISR calls in
this context. The PXE specification requires the ability to make PXE API
calls in interrupt context.
Milestone: Ability to use undi.zpxe (booting from Etherboot-exported PXE
with the rtl8139 driver), with the alternative interrupt handler instead of
trivial_irq_handler, to download and execute an image.
11. Document internal Etherboot driver API changes -- Update documentation
(including the driver writing guide) to include the extra fields that need to be
filled in by probe(), such as IRQ number, and the irq() call. Also update the
skeleton driver.
Milestones: At least two people say that they understand the updated
documentation. At least one person manages to use the updated documentation
to UNDI-enable one of the other drivers.
This is probably another good point to issue a release; we have significant new functionality and a need to stimulate more widespread testing.
STAGE C - Implement UNDI & BC loader support ("PXE ROM")
12. Dummy ROM support -- Implement an extension to mknbi to allow an image
containing an Etherboot program plus a dummy ROM image. Add a debugging
option to the Etherboot UNDI driver to allow it to scan the whole of base
memory for ROM images. This implements structures and code allowing an
Etherboot ROM to function as a PXE ROM, and provides a way of testing
(other than burning a new ROM for each test).
Milestones: Ability to create an "Etherboot + dummy ROM" image containing
undi.zelf plus an Etherboot ROM image for a non-rtl8139 card. Ability to
boot this image using an Etherboot rtl8139.zdsk via a RTL8139 card with
*no* PXE support.
13. UNDI ROM structures -- Add the relevant UNDI headers to the ROM image
to enable undi.zelf to detect it as an UNDI ROM and call the loader entry
point (which will be done in real mode). We need these headers in place
before any of the UNDI loader routines can be called (and hence tested).
Milestones: Ability to create an "Etherboot + dummy ROM" image containing
undi.zelf plus an Etherboot ROM image for a rtl8139 card with PXE export
enabled. This image will henceforth be called "pxe_rom.zelf" for brevity.
When undi.zelf executes, it should detect the Etherboot PXE ROM image and
call the UNDI loader, which will print out the test string.
14. Loader routine -- Implement the UNDI loader routine. The UNDI loader is
needed in order for external programs to install a PXE stack from an Etherboot
ROM, in order to make PXE API calls when not invoked as an NBP from the Etherboot
ROM. It should not enter at main(), but should instead call a routine that will
carry out the usual Etherboot initialisation (relocation, console initialisation
etc.), set up the !PXE structure and return. It should not initialise the NIC.
Milestones: Ability to invoke pxe_rom.zelf (see above) with debugging enabled
in the Etherboot PXE ROM image and see the Etherboot startup strings appearing
when the loader is called. The call to PXENV_START_UNDI should also be
visible.
15. UNDI initialisation routines -- This is the last step in producing an Etherboot
PXE ROM that can be used by undi.zpxe. Implement PXENV_START_UNDI,
PXENV_STOP_UNDI, PXENV_UNLOAD_BASE_STACK and any other load/unload
routines necessary.
Milestones: Ability to burn an Etherboot rtl8139 ROM with PXE export enabled.
Ability to boot undi.zdsk (floppy disk) and have this use the Etherboot ROM to
drive the rtl8139 card via the PXE interface. This is another release stage.
STAGE D - Optional extras
Items within this stage are bonuses; they are not part of the core PXE
stack implementation. Except where indicated, they can be implemented in
any order.
16. UNDI enable all drivers -- Add the irq() function to all Etherboot drivers,
as documented above.
17. PXE DHCP options and ProxyDHCP -- Ensure that we send all options specified
by the PXE spec in our DHCPDISCOVER and DHCPREQUEST packets. Deal with the
PXE-specific options returned by the DHCP server (i.e. store them in a struct
pxe_dhcp_options). When instructed to do so, make further DHCP requests
to the ProxyDHCP server on port 4011. This is done for completeness (PXE ROMs
can function perfectly well without any PXE-specific DHCP options. This effort
would support sites that currently use e.g. ProxyDHCP.
18. Multicast TFTP client and server -- Another "completeness" option. PXE uses
its own variant of multicast TFTP. Etherboot's PXE already has the ability to
obtain a file via multicast (using normal TFTP API to download a file with a URL
indicating a multicast protocol such as SLAM or TFTM). This functionality
would simply add an extra multicast protocol to Etherboot.
19. Remaining UNDI API calls -- Anything that isn't already implemented almost
certainly won't ever be used, but it's worth filling in the gaps.
20. UNDI-on-UNDI operation -- Enable the Etherboot UNDI driver to export itself via
PXE. This will involve damaging the underlying !PXE structure to prevent an NBP
detecting it, then restoring it whenever we make an UNDI API call. (Not really
necessary for any practical reasons other than recursive testing (undi.zpxe on
undi.zpxe on undi.zpxe on undi.zpxe on ...).
======= End summarized plan =======
--
Try: http://rom-o-matic.net/ to make Etherboot images instantly. Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
Cambridge, MA 02139; USA
Voice: (617) 491-6935; Fax: (617) 491-7046
Email: [EMAIL PROTECTED]
Web: http://www.etherboot.org/------------------------------------------------------- This SF.net email is sponsored by: IBM Linux Tutorials. Become an expert in LINUX or just sharpen your skills. Sign up for IBM's Free Linux Tutorials. Learn everything from the bash shell to sys admin. Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click _____________________________________________________________________ Ltsp-discuss mailing list. To un-subscribe, or change prefs, goto: https://lists.sourceforge.net/lists/listinfo/ltsp-discuss For additional LTSP help, try #ltsp channel on irc.freenode.net
