Hi, Guile 2.2 uses ELF to format its object files. ELF objects are composed of *sections* and *segments*. Sections are things like .data, .strtab, and so on; segments contain sections with similar permissions (read-only, read-write, etc). See https://www.gnu.org/software/guile/docs/master/guile.html/Object-File-Format.html for more details.
Parts of different sections refer to each other. For example the read-only .rtl-text code section refers to mutable values in the .data section, by relative offset. It's the linker's job to make all of these connections, and the loader's job to ensure they are valid when the file is loaded. The loader has two paths: an mmap path and a malloc path. The malloc path just slurps the file into memory; the relative references in the file correspond to relative references in memory. With the malloc path, everything is mutable from the operating system's point of view. The problem with the malloc path is that not only is it a bit slower, as it has to page in data that might not be needed, it doesn't share memory between processes. So if the system supports mmap, Guile will use mmap to load its ELF images (.go files). The mmap path likewise just mmaps the file into memory. The operating system will lazily page in data from disk as needed. Writable segments ends up being process-local, but read-only segments can share memory between different processes. This lowers Guile's resident memory footprint, especially when many Guile processes are live. What ends up happening in the mmap case is that the whole file is mapped into memory, then the writable segments are made writable via mprotect(). This only works if the writable segments are page-aligned. And with that prelude out of the way, here's the bug: Guile currently assumes that 4096 is a multiple of the page size. So the mprotect fails to make the segment writable, and runtime relocations that mutate the segment cause segmentation faults. One solution to this issue would be to choose target-specific page sizes. This is still a little tricky; on amd64 for example, systems commonly have 4KB pages, but they are allowed by the ABI to have any multiple-of-2 page size up to 64 KB. On Cygwin, pages are 4kB but they can only be allocated 16 at a time. MIPS and ARM64 can use 64K pages too and that's not uncommon. At the current time, in Guile we would like to reduce the number of binaries we ship to the existing 32-or-64-bit and big-or-little-endian variants, if possible. It would seem that with the least-common-multiple of 64 KB pages, we can do that. See https://github.com/golang/go/issues/10180 for a discussion of this issue in the Go context. So my proposal is to increase the page size to which we align our segments to 64KB. This will increase the size of our .go files, but not the prebuilt/ part of the tarball as that part of the file will be zeroes and compress well. Additionally on a system with 4KB pages, the extra padding will never be paged in, nor read from disk (though it causes more seeking etc so on spinning metal it's a bit of a lose). On many 64-bit platforms, binutils currently defaults to aligning segments on 2MB boundaries. It does so by making the file and the memory images not the same: the pages are all together on disk, but then when loading, the loader will mmap a region "memsz" large which might be greater than the file size, then map segments into that region. I would like to avoid this complication for now. We can consider adding it in the future in a compatible way in 2.2 if it is important. Thoughts welcome :) I am going to give this a go now. Andy
