Greetings, and thank you as always for your wonderful note and work! I was meaning to write to you, as over the weekend I have been committing many of these changes in parallel working with an ACL2 author who has interest in the mac port. I think we are in a position to harden this build quickly.
"Chun Tian (binghe)" <binghe.l...@gmail.com> writes: > Hello, > > yesterday I've sent to MacPorts a port definition for "gcl27" [1], together > with > some macOS specific patches. I'm physically using macOS Big Sur (11.7.10) on > an > Intel Mac, and currently GCL 2.7 doesn't build on ARM Mac yet, so perhaps very > few people can actually use this "gcl27" package at this moment. > I would love to see some detail on the Mac ARM situation, which from my understanding should be as well supported as Linux arm, but apparently there are other difficulties. I am also told Rosetta will enable the x86_64 build on Mac arm. Please enlighten/correct if necessary. > [1] https://github.com/macports/macports-ports/pull/28141 > > Anyway, I have GCL 2.7.1 installed at /opt/local by a locally built MacPorts > package, and now I can start gcl in both ANSI and CLtL1 (by default) modes, > aligned with the Debian testing "gcl27" package (there's no support of > GCL_PROF). > Fantastic! Yes this lack of binutils support is part of the same 'ld' headache I describe below. > Beside the official patch (upstream-small-release-errata), I need to modify > some > other places in the source code to get the build done. The main purpose of > this > email is to explain these patches (they are in attach), to see if some of them > can be part of the upstream source code. > > 1. o/sfaslmacho.c (see section_64.patch in attach) > > At Line 212 there's the following line accessing the "sh_addr" and "sh_size" > members of the "section_64" structure (related to Mach-O format): > > bzero((void *)sec->sh_addr,sec->sh_size); > > But the compiler complains that no such members exist. I think maybe the > following modification should be done: > > bzero((void *)sec->addr,sec->size); Agreed, this is already in master. > > 2. Makefile.in|am (part of binutils.patch) > > The macOS linker doesn't support "-z" option (by calling gcc with > "-Wl,-z,relro"). I checked documentation of "-z" option of "ld" on Linux and > found it's related to ELF format, thus perhaps on macOS we don't need it? (by > deleting this option the building process can continue.) > Agreed, this has been moved to configure.ac where it belongs in master. > 3. Makefile.in|am (part of binutils.patch) > > Some shell commands used in this file involve the grammar <(...), which is > beyond the vanilla (BSD?) "/bin/sh" on macOS. I had to set "SHELL = /bin/bash" > at the beginning of Makefile.in|am and remove the default generated "SHELL = > /bin/sh" in the middle. > Agreed, I've rewritten this to avoid that syntax in master. Hopefully the standard shell will work now for you. > 4. xbin/ar_merge (part of binutils.patch) > > The macOS "ar" command, unfortunately, doesn't support "--output" option, > therefore "ar x" can only extract the archive to the current directory. I > ever > tried to depend on GNU binutils (available in MacPorts) but it seems that > macOS > linker doesn't understand its output format. After some experiments I wrote > the > following working "ar_merge" achieving the same goal but does't rely on "ar > --output": > Agreed, something equivalent in master. Comments welcome as the code is slightly different. > +fullfile= > +filename= > + > TMPDIR=$(mktemp -d) > while [ $# -gt 0 ] ; do > case $(basename $1) in > *.o) cp $1 $TMPDIR;; > *.go) cp $1 $TMPDIR/$(echo $(basename $1)|sed 's,\.go,.o,g');; > - *.a) ar x $1 --output $TMPDIR;; > + *.a) > + fullfile=$1; > + filename="${fullfile##*/}"; > + cp $fullfile $TMPDIR; > + pushd $TMPDIR; > + ar -x $filename; > + rm $filename; > + popd;; > > Regards, > > Chun Tian > > > I am surprised you did not also need 1) The modification to Makefile.am to append to C_INCLUDE_PATH instead of overwrite 2) The modification to lsp/gcl_lr.lsp to omit the popcountl builtin which turns out to be a library call instead, at least on my system. But the real meat of this note regards the memory management. We ran into a failure as the user has 64G installed, and we have a very shortsighted compiled in maximum of 32Gb in unexmacosx.c. In general, GCL goes to some length to lower its starting text address, to *append* new data to the data section produced by the original call to ld producing the raw image, and to reserve the high memory range for the C stack and immediate fixnums (For any interested there is also a configure option to have the immediate fixnum range straddle 0 to avoid the subtraction, but I have never found a convincing payoff for this compared to the much larger range and lower allocation overhead available in high memory.) Interested Mac users should be aware of otool and vmmap which are the only means I have found to figure out what is going on. But essentially it is this -- on Mac the dynamic linker and its attached libraries sits right on top of the end of the data section, and there is no linker script nor any other facility I can see with which to move it. (Alternatively, there may be a creative way to define a large uninitialized array in C and arrange to have it placed at the end of the data section and start mmap from there.) This leads to the following suboptinal situation. In the raw image, we gap the heap up above the dynamic area with an unspecified address call to mmap. Then when dumping via unexec, we encode a large but fixed and arbitrary vmsize of 32Gb to the _HEAP, so that on startup we have the gap we want between the end of _HEAP and the new dynamic memory area for the _HEAP to grow into. This leaves a hole occupied by the original dynamic area, but more worryingly hardcodes a maximum heap into the image which is independent of the memory available on the user's system. A long time ago GCL hardcoded 'maxpages' into the image, and this lead to all kinds of difficulties as that image was moved and distributed to various systems with different installed memories. We need and want a memory agnostic image which will adapt to the runtime memory profile. Of course all of these numbers are integers in a fixed range, so nothing is truly unbounded. The practical approach is to increase the vmsize to several times the maximum memory available on apple systems, and I intend to commit something to this effect shortly. But the right thing to do is to use a linker script to move the dynamic area out of the way in the first place. Alas there are no linker scripts available (apparently), and no facility in the 'load commands' of mach-o to do this (that I can see). Mac experts are *very* welcome to correct me! Beyond this, we now have 3 independent emulations of sbrk (linux, mac, windows) which is error prone and totally unnecessary. I am testing the linux code on the mac and it works fine. There is just no variable _end placed at the end of the data section as on linux to determine where to start, and even if there was it would not be usable because of the dynamic area issue. So presuming the latter is a permanent problem I will need a way to get to an address above with which to start the sbrk. Once done there will be a 3 line change to 386-macosx.h to use the linux sbrk. Take care, -- Camm Maguire c...@maguirefamily.org ========================================================================== "The earth is but one country, and mankind its citizens." -- Baha'u'llah
