On Sun, 19 Jun 2005 16:23:16 +1000, Darren Tucker wrote:
>Chris Zakelj wrote:
>> I'm curious as to how programs actually get ported from one OS to
>> another,
>
>Yes, some techniques make the job easier, but it depends on what the
>program does and whether you're doing a one-way port or an ongoing port.
> The following aren't necessarily the only ways to do porting, but it's
>how the Open* Portable projects are done.
>
>If it's a one-way port (ie the port will be done once and thereafter
>maintained separately), the usual method is to just change what needs
>changing to suit the target platform. This is effectively what happened
>when the original ssh-1.2.12 code was 'ported' to OpenBSD.
>
>In the case of OpenSSH and OpenNTPD Portable, they're ongoing ports (ie
>changes are regularly sync'ed from OpenBSD's to Portable's). There are
>2 main ways to accomplish this: sprinkle the code with #ifdefs or
>implement the missing functionality in a compatibilty layer.
>
>In the Portable projects, the first preference is leave the common code
>alone and implement the required functionality in a portability layer.
>This has the advantage of keeping the common code clean and if done
>properly the components are reusable. (A number of functions used by
>OpenNTPD Portable came unmodified from OpenSSH Portable). Sometimes
>that's not possible or more effort than it's worth, so in those cases an
>#ifdef is used which imposes an ongoing maintenance cost (ie next time a
>change is made in that area in the main code, you'll have to manually
>resolve conflict when syncing changes).
>
>For example: OpenNTPD Portable was ported to run on QNX4 (a POSIXish
>embedded system) by Anthony O.Zabelin. The 2 main missing pieces were
>the adjtime() and poll() calls. Simplifying somewhat, the code that
>used adjtime looked like this:
>
> d_to_tv(d, &tv);
> if (adjtime(&tv, NULL) == -1)
> log_warn("adjtime failed");
>
>If we had used the #ifdef technique, that would have changed to
>something like this:
>
>#ifndef __QNX__
> d_to_tv(d, &tv);
> if (adjtime(&tv, NULL) == -1)
> log_warn("adjtime failed");
>#else
> usec = (int)(d * 1000000);
> if (qnx_adj_time(usec, ADJUST_RATE, NULL, NULL) == -1)
> log_warn("qnx_adj_time failed");
>#endif
>
>Now one or two of those aren't too bad, but it rapidly becomes difficult
>to follow once you add a few more to the same piece of code.
>
>Instead, Anthony wrote a stand-alone replacement adjtime() function
>which is in the portability layer (openbsd-compat/port-qnx.c). This had
>a higher initial cost (it's 23 lines of code in a single function plus a
>Makefile change instead of 6 lines listed above) but it leaves the main
>code unchanged. It can also be tested separately and is reusable.
>
>I took the same approach with poll() and built a replacement on top of
>select(), which QNX4 did have. Hey presto, it now worked on QNX4, and
>the codebase is no harder to maintain.
>
>> and if certain directions are easier than others.
>
>In general, the difficulty is directly proportional to how different the
>target platform is compared to the platforms already supported in the
>area in which the program operates. In most current OSes there's a slow
>convergance toward common APIs for standard languages so if you stick to
>those standard APIs you life will be easier.
>
>Newer/more featureful -> older/less featureful is usually harder than
>the other way around unless the program was originally written to stick
>to a common subset.
>
>Beyond that it depends on the program. Porting a simple text filter
>from a bleeding-edge Linux to 10-year old BSD is likely to be simple,
>but other programs may be difficult to impossible.
>
>> That is, how does one figure out what needs to be changed in order to
>> make OpenNTPD work on Linux?
>
>I had the advantage of having worked on Portable OpenSSH for a couple of
>years so had a reasonable idea what to expect, so for OpenNTPD I just
>copied the code onto a Linux box, hacked the BSDisms out of the Makefile
>and tried to compile it. This highlighted some obvious problems (eg
>missing strl* functions, the lack of sa_len in struct sockaddr). I
>fixed these (stole the strl* functions from OpenSSH and changed
>sa->sa_len into SA_LEN(sa)) and tried again. After a few iterations of
>this process it compiled and after a couple more, amazingly enough, it
>worked. At that point I added basic autoconf support, put a tarball on
>my web page and mentioned it on [EMAIL PROTECTED]
>
>After that, other folks and myself repeated the process on other
>platforms, slowly expanding the list that it would run on. (The
>platform list on openntpd.org is in chronological order, earliest first).
>
>> Is it generally easier to move a program from $some_bsd
>> to $some_other_os, or from $some_other_os to $some_bsd?
>
>Depends on the type of program, and in particular what OS-level services
>it uses.
>
>OpenSSH, for example, has to deal with user authentication for which
>there is a large amount of variance between platforms, so the diff
>between OpenBSD's and Portable's is large.
>
>OpenNTPD has to deal with far less variance between platforms so the
>diff is much smaller. (adjtime() is common and the interface is simple,
>but if/when it starts compensating for systematic clock skew then that
>will introduce a significant amount of platform-specific code, however
>most of that can be hidden in the compatibility layer.)
>
>> How would you even begin to port something like OpenSSH to a non-Unix
>> system like Windows?
>
>I would say you would need a POSIX-like compatibility layer (eg external
>like Cygwin or implement your own), otherwise you would probably have to
>do a one-way port. In that case you could keep the protocol code as is,
>but you would probably need to replace large chunks of the rest.
>
>> Does the chosen language (C, C++, Java, etc) make a difference
>> in difficulty?
>
>I don't use C++ or Java so I won't comment on them, but IMO the language
>can a make significant difference, but only for some things.
>
>It is possible to write portable C. In the case of OpenSSH, most of the
>platform-dependant code is there because the operating system requires
>it, not the language, and unless the language provides an abstraction
>for exactly what you're trying to do then the choice of language makes
>little difference.
>
>For example (and I'm oversimplifying here), validating a user's
>password: on a traditional Unix system you encrypt the password and
>compare against what getpwnam() returns. If your system has shadow
>passwords, though, you need to use getspnam() since the encrypted
>password isn't returned by getpwnam() on those. Oh, and some really old
>systems based on SecureWare have another function (getprpwnam). (This
>is ignoring platform-specific functions and the attempts to standardize
>this in libraries such as BSD auth or PAM, which have varying degrees of
>success.)
>
>If a language provided a "authenticate_user(user, password)" then that
>would help in this case, but most don't seem to (and there are many
>other examples of platform-dependant things you would need to do besides
>this one).
>
>Anyway, some guidelines to avoid common traps for portable C:
>
>1) write for correctness and clarity.
>
>2) try to stick to standard functions (eg POSIX).
>
>3) be prepared to implement your own replacements for non-standard
>functions you use.
>
>3b) or standard functions that aren't available on you target platform.
>
>3c) or standard functions that are broken on your target platform.
>
>4) if possible put all the system #includes in a single header and
>include that from all of your source files. Headers vary quite a bit
>and it's better if you only have to deal with all that variance once.
>
>5) use the datatypes you're supposed to. eg if you need to store a 32
>bit value, use "u_int32_t" not "unsigned int", if you're using it in a
>signal handler use "sig_atomic_t" not "int". Check and typedef it
>yourself if your platform doesn't.
>
>6) Turn on all the compiler warnings and fix what it warns about. Many
>are potential portability problems, even if they haven't bitten you yet.
>
>[This list isn't exhaustive, I'm sure other folks will be able to add to
>it.]
>
>> When I've built from ports, I can see make files doing
>> OS detection, but from there (not being a very good coder), I can't
>> really make out how it changes the code based on that. Any
>> recommendations for "casual programmer" books would be cool...
>
>A place to start would be this diff:
>http://www.zip.com.au/~dtucker/openntpd/patches/ntpd-vs-openbsd.diff
>It shows the (small) changes to the OpenBSD ntpd code (3.6.1) to make it
>"Portable". The remainder of the changes are in files that are only in
>Portable (take a look in openbsd-compat/ in the OpenNTPD portable
>distribution).
>
>After that, I suggest downloading the OpenBSD-specific ntpd and the
>equivalent Portable one and comparing them (diff -ru is your friend).
>It's small enough that it ought to be understandable but it's a real
>application that runs on 9 platforms (so far :-).
>
>--
>Darren Tucker (dtucker at zip.com.au)
>GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
> Good judgement comes with experience. Unfortunately, the experience
>usually comes from bad judgement.
>
>
I wouldn't have had a reason to ask the question you answered - I think
I'm getting to be a bit too rusty to do stuff like that these days.
That said, I have to congratulate you on a wonderfully clear
description of the process as seen by a craftsman. Although it wouldn't
get me back to doing the same type of task as the one described, the
old synapses recognised that you were handing out a valuable lesson to
the up-and-coming coders we (or rather, non-OBSD users) depend on.
On behalf of the community -Thanks mate! Bloody well written!
In the beginning was The Word
and The Word was Content-type: text/plain
The Word of Rod.
Do NOT CC me - I am subscribed to the list.
Replies to the sender address will fail except from the list-server.
