Hello, list! I thought I should write a few words on some information I’ve gleaned trying to build picolisp on some of the BSDs. There are people on this list with more experience than I with both BSD and picolisp; so please comment upon what I have to say, and weigh in where you feel, or know, that it is necessary, for the benefit of everyone. This is a learning experience for me. I am not in a position to write authoritatively. I will try to accurately report my findings, but please especially point out where I am wrong or being misleading. (I would have done this unintentionally, of course.)
*Bottom Line* Starting from the picolisp 3.1.9.13 source — the latest as of this past weekend (more about this below) — I was able to successfully build and unit-test pil32 on the following BSDs: FreeBSD 10.1, OpenBSD 5.6, and NetBSD 6.1.5. Also, I was able to successfully build and unit-test pil64 on FreeBSD 10.1. (AFAIK, only FreeBSD, among the three, has bootstrap support (with pil32) to build pil64, and I only looked at the bootstrapping option for building pil64.) The following sections just spell out some detail about the builds. A quick note before that: on each BSD box in question, I had to install gmake. This shouldn’t be a surprise — picolisp builds need GNU make, and the BSDs’ stock make is BSD, not GNU. *FreeBSD 10.1* I was aiming for a pil64 build in this machine, so I started with a 64-bit (amd64) FreeBSD 10.1 box. First, I had to build pil32 (for the bootstrap build). On a fresh FreeBSD 10 box you won’t have gcc, so you need to install that. I used pkg for that (i.e. I didn’t build gcc from ports). Its interface is installed in /usr/local/bin/gcc48, by the way. But, why install gcc? After all, clang (the stock compiler) is a “drop-in replacement for gcc“, right? Wrong. If you try to build pil32 with clang, you will be greeted with the following errors. apply.o.log:apply.c:15:40: error: fields must have a constant size: 'variable length array in structure' extension will never be supported apply.o.log:apply.c:67:43: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:91:37: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:159:40: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:299:34: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:663:37: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:698:34: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:736:37: error: fields must have a constant size: 'variable length array in structure' extension will never be supported flow.o.log:flow.c:786:37: error: fields must have a constant size: 'variable length array in structure' extension will never be supported main.o.log:main.c:720:34: error: fields must have a constant size: 'variable length array in structure' extension will never be supported Yes, that’s right folks, clang does not support dynamically sized arrays and never will (!) as unequivocally stated by their error message: “extension will never be supported“ (“never”? really?). (Take that picolispers! This reminds me of the taunting Frenchman in Monty Python’s Holy Grail: “No chance, English bedwetting types!”) But gcc does support this; so, back to gcc. The objects build fine with gcc, but then we run into a problem in the link step. gcc48 -o ../bin/picolisp -m32 -rdynamic main.o gc.o apply.o flow.o sym.o subr.o big.o io.o net.o tab.o -lm /usr/local/bin/ld: skipping incompatible //usr/lib/libm.so when searching for -lm /usr/local/bin/ld: skipping incompatible //usr/lib/libm.a when searching for -lm /usr/local/bin/ld: cannot find -lm /usr/local/bin/ld: skipping incompatible /usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/libgcc.a when searching for -lgcc /usr/local/bin/ld: skipping incompatible //usr/lib/libgcc.a when searching for -lgcc /usr/local/bin/ld: cannot find -lgcc /usr/local/bin/ld: skipping incompatible /usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/../../../libgcc_sso when searching for -lgcc_s /usr/local/bin/ld: skipping incompatible //usr/lib/libgcc_s.so when searching for -lgcc_s /usr/local/bin/ld: cannot find -lgcc_s /usr/local/bin/ld: skipping incompatible /lib/libc.so.7 when searching for /lib/libc.so.7 /usr/local/bin/ld: cannot find /lib/libc.so.7 /usr/local/bin/ld: skipping incompatible /usr/lib/libc_nonshared.a when searching for /usr/lib/libc_nonshared.a /usr/local/bin/ld: cannot find /usr/lib/libc_nonshared.a /usr/local/bin/ld: skipping incompatible /usr/lib/libssp_nonshared.a when searching for /usr/lib/libssp_nonshared.a /usr/local/bin/ld: cannot find /usr/lib/libssp_nonshared.a Oops. gcc cannot find the 32-bit libraries. But they do exist on the stock system. Rrrr. This gcc, from pkg, is not built for multilib support. I might be able to build gcc in ports with multilib support, but I had another thought: can clang do just the link step? After all, it’s the stock compiler and the stock system already has the 32-bit libraries there. And yes, clang indeed does the job of finding them for the link step. So, you’d only need to change a line in src/Makefile at the link step to call clang instead of gcc, and you should have a successful pil32 build on 64-bit FreeBSD 10. Now, this brings us to *a change introduced in picolisp last weekend*, which is that src/Makefile has been changed so that it works for FreeBSD 10 in the way described above “out of the box” with the new version, 3.1.9.13. It has also been slightly refactored so that it is backward-compatible as regards the other systems (e.g. Linux, Solaris), i.e. it will work *the same way as before* on these systems, namely that gcc is (still) the interface for both the object build and the link step. Please be aware of this change; however, it shouldn’t give you a problem. Alex has already tested the change on some Linuxen, and I’ve already tested it on all the BSDs mentioned here. After building pil32, building pil64 is very easy (and fast!). Also, please note that I did not try building pil32 on FreeBSD 10.1 i386. This case should probably be checked, at least for completion’s sake. Finally, here is a log of the “out of box” experience building picolisp. The new src/Makefile in version 3.1.9.13 takes care the issues we just discussed. $ echo "$(uname -s) $(uname -r) $(uname -p)" FreeBSD 10.1-RELEASE amd64 $ cd ~/distros $ curl -O http://software-lab.de/picoLisp.tgz && mv picoLisp.tgz picoLisp-3.1.9.13.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 894k 100 894k 0 0 1393k 0 --:--:-- --:--:-- --:--:-- 1399k $ cd ~/builds $ tar zxf ~/distros/picoLisp-3.1.9.13.tgz && mv picoLisp picoLisp-3.1.9.13 $ cd picoLisp-3.1.9.13 $ (cd src; gmake) gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' main.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' gc.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' apply.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' flow.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' sym.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' subr.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' big.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' io.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' net.c gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' tab.c mkdir -p ../bin ../lib clang -o ../bin/picolisp -m32 -rdynamic main.o gc.o apply.o flow.o sym.o subr.o big.o io.o net.o tab.o -lm strip ../bin/picolisp gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' ext.c clang -o ../lib/ext -m32 -shared -export-dynamic ext.o -lcrypt strip ../lib/ext gcc48 -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"FreeBSD"' ht.c clang -o ../lib/ht -m32 -shared -export-dynamic ht.o strip ../lib/ht $ # test pil32 $ ./pil lib/test.l $(/bin/pwd) -bye + OK $ (cd src64; gmake) $ # test pil64 $ ./pil lib/test.l $(/bin/pwd) -bye + OK $ ./pil -version -bye 3.1.9.13 *OpenBSD 5.6* I have a server running OpenBSD 5.6 (amd64), so I thought I’d give that a try to boostrap-build pil64. So, I attempted to build pil32 first. This did not work. AFAIK there is no stock multilib support on this OS. (I couldn’t find any 32-bit libraries on the system.) There may be a way to build gcc from sources that supports multilib, but I’m not aware of it. (Does anyone know?) I think I recall Mike Pechkin, our resident OpenBSD expert, telling me on IRC that this is not possible. Too bad, because I really like OpenBSD and would love to run pil64 on it someday. I was able, however, to successfully build and unit-test pil32 on OpenBSD 5.6 (i386). (Actually, originally two of the unit tests hung on a system network call, but Mike P told me on IRC how to configure my network on the box to solve this problem; then all the tests passed. He understands the net stack. I don’t. It’s like “deep magic” for me.) Here’s the build log on an i386 OpenBSD 5.6. $ echo "$(uname -s) $(uname -r) $(uname -p)" OpenBSD 5.6 i386 $ cd ~/distros $ ls picoLisp-3.1.5.tgz picoLisp-3.1.6.tgz picoLisp-3.1.7.tgz picoLisp-3.1.8.tgz picoLisp-3.1.9.12.tgz picoLisp-3.1.9.tgz $ curl -O http://software-lab.de/picoLisp.tgz && mv picoLisp.tgz picoLisp-3.1.9.13.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 894k 100 894k 0 0 257k 0 0:00:03 0:00:03 --:--:-- 277k $ cd ~/builds $ tar zxf ~/distros/picoLisp-3.1.9.13.tgz && mv picoLisp picoLisp-3.1.9.13 $ cd picoLisp-3.1.9.13 $ (cd src; gmake) gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' main.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' gc.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' apply.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' flow.c flow.c: In function 'doTick': flow.c:1583: warning: cast to pointer from integer of different size flow.c:1584: warning: cast to pointer from integer of different size gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' sym.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' subr.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' big.c big.c:7:1: warning: "MAX" redefined In file included from /usr/include/netdb.h:93, from pico.h:30, from big.c:5: /usr/include/sys/param.h:217:1: warning: this is the location of the previous definition gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' io.c io.c: In function 'doEcho': io.c:2261: warning: 'op' may be used uninitialized in this function gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' net.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' tab.c mkdir -p ../bin ../lib gcc -o ../bin/picolisp -m32 -rdynamic -Wl,-E main.o gc.o apply.o flow.o sym.o subr.o big.o io.o net.o tab.o -lm main.o(.text+0xa35): In function `sharedLib': : warning: strcpy() is almost always misused, please use strlcpy() big.o(.text+0x3971): In function `numToSym': : warning: sprintf() is often misused, please use snprintf() strip ../bin/picolisp gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' ext.c gcc -o ../lib/ext -Wl,-E -Wl,-shared ext.o -lcrypto strip ../lib/ext gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"OpenBSD"' ht.c gcc -o ../lib/ht -Wl,-E -Wl,-shared ht.o strip ../lib/ht $ ./pil lib/test.l $(/bin/pwd) -bye + OK $ ./pil -version -bye 3.1.9.13 C *NetBSD 6.1.5* On a NetBSD 6.1.5 (amd64) box, I was able to build (and unit-test) pil32 with no issues. The stock compiler is gcc and has multilib support. Nice. As an aside, this might be a good target for a pil64 bootstrap build someday. Has anyone tried this? I did not try building pil32 on NetBSD 6.1.5 i386. For completion’s sake, this case should probably be checked also. Here’s the build log on that machine. $ echo "$(uname -s) $(uname -r) $(uname -p)" NetBSD 6.1.5 x86_64 $ cd ~/distros $ curl -O http://software-lab.de/picoLisp.tgz && mv picoLisp.tgz picoLisp-3.1.9.13.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 894k 100 894k 0 0 120k 0 0:00:07 0:00:07 --:--:-- 148k $ cd ~/builds $ tar zxf ~/distros/picoLisp-3.1.9.13.tgz && mv picoLisp picoLisp-3.1.9.13 $ cd picoLisp-3.1.9.13 $ (cd src; gmake) gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' main.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' gc.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' apply.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' flow.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' sym.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' subr.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' big.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' io.c io.c: In function 'doEcho': io.c:2261:50: warning: 'op' may be used uninitialized in this function gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' net.c gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' tab.c mkdir -p ../bin ../lib gcc -o ../bin/picolisp -m32 -rdynamic main.o gc.o apply.o flow.o sym.o subr.o big.o io.o net.o tab.o -lm strip ../bin/picolisp gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' ext.c gcc -o ../lib/ext -m32 -shared -export-dynamic ext.o -lcrypto strip ../lib/ext gcc -c -O2 -pipe -falign-functions=32 -fomit-frame-pointer -fno-strict-aliasing -W -Wimplicit -Wreturn-type -Wunused -Wformat -Wuninitialized -Wstrict-prototypes -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -m32 -D_OS='"NetBSD"' ht.c gcc -o ../lib/ht -m32 -shared -export-dynamic ht.o strip ../lib/ht $ ./pil lib/test.l $(/bin/pwd) -bye + OK $ ./pil -version -bye 3.1.9.13 C
