Re: i386 kernel relinking
On Tue, May 19, 2020 at 05:21:59PM +0200, Stefan Sperling wrote: > On Fri, Apr 10, 2020 at 10:25:14AM -0400, Nick Holland wrote: > > On 2020-04-10 10:10, Stefan Sperling wrote: > > > On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote: > > > FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage > > > to relink a kernel (on 6.6 + syspatches). > > > > Whoops. Guess I should have mentioned, that was -current, as of > > yesterday > > OpenBSD 6.7-beta (GENERIC.MP) #110: Thu Apr 9 01:20:52 MDT 2020 > > dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP > > real mem = 334970880 (319MB) > > avail mem = 313077760 (298MB) > > > > and probably a couple weeks ago for the real (old) hw. > > > > I'm curious if your Soekris can handle 6.7-beta. > > > > Nick. > > It's been fixed in 6.7 release! Thank you!!! Awesome!!!
Re: i386 kernel relinking
On Fri, Apr 10, 2020 at 10:25:14AM -0400, Nick Holland wrote: > On 2020-04-10 10:10, Stefan Sperling wrote: > > On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote: > > FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage > > to relink a kernel (on 6.6 + syspatches). > > Whoops. Guess I should have mentioned, that was -current, as of > yesterday > OpenBSD 6.7-beta (GENERIC.MP) #110: Thu Apr 9 01:20:52 MDT 2020 > dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP > real mem = 334970880 (319MB) > avail mem = 313077760 (298MB) > > and probably a couple weeks ago for the real (old) hw. > > I'm curious if your Soekris can handle 6.7-beta. > > Nick. It's been fixed in 6.7 release! This was indeed a problem on my alix and soekris machines for a while. And I could also reproduce it in a 6.7-beta low memory VM as you did. I attempted to debug ld.lld for a while but eventually gave up (debugging the linker is quite painful). I did not come to any conclusion but did notice that ld.lld uses mmap to write ELF data to the output file. That file always ended up being corrupt under low memory conditions. The old ld.bfd linker uses stdio and worked OK. With 6.7 kernel relinking with little amounts of memory suddenly started working again. To make sure I'm not dreaming I bisected the commit which fixed it. Turns out this problem was fixed by beck@ in the commit below. I can even relink kernels on i386 in 64MB of RAM now. Didn't test lower than that. Case closed. Just upgrade to 6.7 and it'll be fine :-) CVSROOT:/cvs Module name:src Changes by: b...@cvs.openbsd.org2020/04/28 20:25:48 Modified files: sys/kern : vfs_bio.c Log message: Ensure that if we are doing a delayed write with a NOCACHE buffer, we clear the NOCACHE flag, since if we are doing a delayed write the buffer must be cached or it is thrown away when the "write" is done. fixes vnd on mfs regress tests. ok kettenis@ deraadt@
Re: i386 kernel relinking
FWIW, the GNU linker can reorder the kernel on i386-current
with 256MB RAM:
# env LD=ld.bfd /usr/libexec/reorder_kernel
---
relink.log:
(SHA256) /bsd: OK
LD="ld.bfd" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
ld.bfd -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o
${OBJS}
textdatabss dec hex
11916299262796 1101824 13280919caa697
mv newbsd newbsd.gdb
ctfstrip -S -o newbsd newbsd.gdb
rm -f bsd.gdb
mv -f newbsd bsd
install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
Kernel has been relinked and is active on next reboot.
SHA256 (/bsd) = 3c9cca9da06acdc92270f6f0e68b57447881a01e3e2584a0086291efb5033ba7
---
On Fri, Apr 10, 2020 at 10:52:37AM -0600, Theo de Raadt wrote:
> I am succesfully relinking kernels on a landisk with 128MB of ram.
>
> I think this conversation is ridiculous:
>
> If a machine is too small, then it is too small. Do I have to paypal
> a $0.05 to some users?
>
> Nick Holland wrote:
>
> >
> >
> >
> > On 2020-04-10 10:10, Stefan Sperling wrote:
> > > On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote:
> > >> Question about kernel randomization and relinking...
> > >>
> > >> It seems to take a fair amount of RAM, at least for systems that
> > >> are forced to run i386. And I mean real RAM -- swap doesn't seem
> > >> to cut it.
> > >>
> > >> I discovered that several machines I was intending on using for
> > >> minimal purposes just couldn't complete relinking. So I built a
> > >> VM and started playing with the RAM.
> > >>
> > >> Built with 1G RAM, default was a 1.2G swap, worked fine.
> > >> Reduced to 256MB RAM, Kernel failed to relink. As with my old
> > >> junk.
> > >>
> > >> The magic number seemed to be between 320MB (failed) and 384MB
> > >> (worked) of RAM. Ok, fine.
> > >
> > > FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage
> > > to relink a kernel (on 6.6 + syspatches).
> >
> > Whoops. Guess I should have mentioned, that was -current, as of
> > yesterday
> > OpenBSD 6.7-beta (GENERIC.MP) #110: Thu Apr 9 01:20:52 MDT 2020
> > dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
> > real mem = 334970880 (319MB)
> > avail mem = 313077760 (298MB)
> >
> > and probably a couple weeks ago for the real (old) hw.
> >
> > I'm curious if your Soekris can handle 6.7-beta.
> >
> > Nick.
> >
> >
> > >
> > > # ls -l relink.log
> > > -rw-r--r-- 1 root wheel - 507B Apr 10 13:33 relink.log
> > > # cat relink.log
> > > (SHA256) /bsd: OK
> > > LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
> > > ld -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o
> > > ${OBJS}
> > > textdatabss dec hex
> > > 11815507267748 1101824 13185079c93037
> > > mv newbsd newbsd.gdb
> > > ctfstrip -S -o newbsd newbsd.gdb
> > > rm -f bsd.gdb
> > > mv -f newbsd bsd
> > > install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
> > >
> > > Kernel has been relinked and is active on next reboot.
> > >
> > > SHA256 (/bsd) =
> > > a940ce989d708e5b87a1186ee81bd624066baeabe67b8405b52e4fa2988b565
> > >
> > >
> > > # dislabel -pm wd0
> > > #size offset fstype [fsize bsize cpg]
> > > a: 353.0M 64 4.2BSD 2048 16384 5624 # /
> > > b: 511.1M 722944swap# none
> > > c: 15280.0M0 unused
> > > d: 444.8M 1769728 4.2BSD 2048 16384 7116 # /tmp
> > > e: 607.7M 2680576 4.2BSD 2048 16384 9685 # /var
> > > f: 1703.0M 3925216 4.2BSD 2048 16384 12958 # /usr
> > > g: 505.8M 7412896 4.2BSD 2048 16384 8060 #
> > > /usr/X11R6
> > > h: 1632.9M 8448736 4.2BSD 2048 16384 12958 #
> > > /usr/local
> > > i: 1381.2M 11792960 4.2BSD 2048 16384 12958 #
> > > /usr/src
> > > j: 5282.4M 14621632 4.2BSD 2048 16384 12958 #
> > > /usr/obj
> > > k: 2850.9M 25439936 4.2BSD 2048 16384 12958 # /home
> > >
> >
on "underpowered" machines (was: Re: i386 kernel relinking)
theo wrote: > I think this conversation is ridiculous: > > If a machine is too small, then it is too small. Do I have to paypal > a $0.05 to some users? Yes, me! Pretty please? *kitty eyes* Seriously: sometimes, a machine is perfectly adequate, except for one task that eats disproportionate amounts of resources. Questioning the apropriateness of that task would seem to be a reasonable step to undertake, before running to the store waving your credit card (not to speak of the waste problem). Apart from the fact that many people on this planet *are* really-terribly-out-of-pocket, there are still other issues than money. --zeurkous. -- Friggin' Machines!
Re: i386 kernel relinking
I am succesfully relinking kernels on a landisk with 128MB of ram.
I think this conversation is ridiculous:
If a machine is too small, then it is too small. Do I have to paypal
a $0.05 to some users?
Nick Holland wrote:
>
>
>
> On 2020-04-10 10:10, Stefan Sperling wrote:
> > On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote:
> >> Question about kernel randomization and relinking...
> >>
> >> It seems to take a fair amount of RAM, at least for systems that
> >> are forced to run i386. And I mean real RAM -- swap doesn't seem
> >> to cut it.
> >>
> >> I discovered that several machines I was intending on using for
> >> minimal purposes just couldn't complete relinking. So I built a
> >> VM and started playing with the RAM.
> >>
> >> Built with 1G RAM, default was a 1.2G swap, worked fine.
> >> Reduced to 256MB RAM, Kernel failed to relink. As with my old
> >> junk.
> >>
> >> The magic number seemed to be between 320MB (failed) and 384MB
> >> (worked) of RAM. Ok, fine.
> >
> > FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage
> > to relink a kernel (on 6.6 + syspatches).
>
> Whoops. Guess I should have mentioned, that was -current, as of
> yesterday
> OpenBSD 6.7-beta (GENERIC.MP) #110: Thu Apr 9 01:20:52 MDT 2020
> dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
> real mem = 334970880 (319MB)
> avail mem = 313077760 (298MB)
>
> and probably a couple weeks ago for the real (old) hw.
>
> I'm curious if your Soekris can handle 6.7-beta.
>
> Nick.
>
>
> >
> > # ls -l relink.log
> > -rw-r--r-- 1 root wheel - 507B Apr 10 13:33 relink.log
> > # cat relink.log
> > (SHA256) /bsd: OK
> > LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
> > ld -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o
> > ${OBJS}
> > textdatabss dec hex
> > 11815507267748 1101824 13185079c93037
> > mv newbsd newbsd.gdb
> > ctfstrip -S -o newbsd newbsd.gdb
> > rm -f bsd.gdb
> > mv -f newbsd bsd
> > install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
> >
> > Kernel has been relinked and is active on next reboot.
> >
> > SHA256 (/bsd) =
> > a940ce989d708e5b87a1186ee81bd624066baeabe67b8405b52e4fa2988b565
> >
> >
> > # dislabel -pm wd0
> > #size offset fstype [fsize bsize cpg]
> > a: 353.0M 64 4.2BSD 2048 16384 5624 # /
> > b: 511.1M 722944swap# none
> > c: 15280.0M0 unused
> > d: 444.8M 1769728 4.2BSD 2048 16384 7116 # /tmp
> > e: 607.7M 2680576 4.2BSD 2048 16384 9685 # /var
> > f: 1703.0M 3925216 4.2BSD 2048 16384 12958 # /usr
> > g: 505.8M 7412896 4.2BSD 2048 16384 8060 #
> > /usr/X11R6
> > h: 1632.9M 8448736 4.2BSD 2048 16384 12958 #
> > /usr/local
> > i: 1381.2M 11792960 4.2BSD 2048 16384 12958 # /usr/src
> > j: 5282.4M 14621632 4.2BSD 2048 16384 12958 # /usr/obj
> > k: 2850.9M 25439936 4.2BSD 2048 16384 12958 # /home
> >
>
Re: i386 kernel relinking
On 2020-04-10 10:10, Stefan Sperling wrote:
> On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote:
>> Question about kernel randomization and relinking...
>>
>> It seems to take a fair amount of RAM, at least for systems that
>> are forced to run i386. And I mean real RAM -- swap doesn't seem
>> to cut it.
>>
>> I discovered that several machines I was intending on using for
>> minimal purposes just couldn't complete relinking. So I built a
>> VM and started playing with the RAM.
>>
>> Built with 1G RAM, default was a 1.2G swap, worked fine.
>> Reduced to 256MB RAM, Kernel failed to relink. As with my old
>> junk.
>>
>> The magic number seemed to be between 320MB (failed) and 384MB
>> (worked) of RAM. Ok, fine.
>
> FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage
> to relink a kernel (on 6.6 + syspatches).
Whoops. Guess I should have mentioned, that was -current, as of
yesterday
OpenBSD 6.7-beta (GENERIC.MP) #110: Thu Apr 9 01:20:52 MDT 2020
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
real mem = 334970880 (319MB)
avail mem = 313077760 (298MB)
and probably a couple weeks ago for the real (old) hw.
I'm curious if your Soekris can handle 6.7-beta.
Nick.
>
> # ls -l relink.log
> -rw-r--r-- 1 root wheel - 507B Apr 10 13:33 relink.log
> # cat relink.log
> (SHA256) /bsd: OK
> LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
> ld -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o
> ${OBJS}
> textdatabss dec hex
> 11815507267748 1101824 13185079c93037
> mv newbsd newbsd.gdb
> ctfstrip -S -o newbsd newbsd.gdb
> rm -f bsd.gdb
> mv -f newbsd bsd
> install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
>
> Kernel has been relinked and is active on next reboot.
>
> SHA256 (/bsd) =
> a940ce989d708e5b87a1186ee81bd624066baeabe67b8405b52e4fa2988b565
>
>
> # dislabel -pm wd0
> #size offset fstype [fsize bsize cpg]
> a: 353.0M 64 4.2BSD 2048 16384 5624 # /
> b: 511.1M 722944swap# none
> c: 15280.0M0 unused
> d: 444.8M 1769728 4.2BSD 2048 16384 7116 # /tmp
> e: 607.7M 2680576 4.2BSD 2048 16384 9685 # /var
> f: 1703.0M 3925216 4.2BSD 2048 16384 12958 # /usr
> g: 505.8M 7412896 4.2BSD 2048 16384 8060 # /usr/X11R6
> h: 1632.9M 8448736 4.2BSD 2048 16384 12958 # /usr/local
> i: 1381.2M 11792960 4.2BSD 2048 16384 12958 # /usr/src
> j: 5282.4M 14621632 4.2BSD 2048 16384 12958 # /usr/obj
> k: 2850.9M 25439936 4.2BSD 2048 16384 12958 # /home
>
Re: i386 kernel relinking
On Fri, Apr 10, 2020 at 09:35:16AM -0400, Nick Holland wrote:
> Question about kernel randomization and relinking...
>
> It seems to take a fair amount of RAM, at least for systems that
> are forced to run i386. And I mean real RAM -- swap doesn't seem
> to cut it.
>
> I discovered that several machines I was intending on using for
> minimal purposes just couldn't complete relinking. So I built a
> VM and started playing with the RAM.
>
> Built with 1G RAM, default was a 1.2G swap, worked fine.
> Reduced to 256MB RAM, Kernel failed to relink. As with my old
> junk.
>
> The magic number seemed to be between 320MB (failed) and 384MB
> (worked) of RAM. Ok, fine.
FWIW, my soekris net5501 with 256MB of RAM and 512MB swap does manage
to relink a kernel (on 6.6 + syspatches).
# ls -l relink.log
-rw-r--r-- 1 root wheel - 507B Apr 10 13:33 relink.log
# cat relink.log
(SHA256) /bsd: OK
LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
ld -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o ${OBJS}
textdatabss dec hex
11815507267748 1101824 13185079c93037
mv newbsd newbsd.gdb
ctfstrip -S -o newbsd newbsd.gdb
rm -f bsd.gdb
mv -f newbsd bsd
install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
Kernel has been relinked and is active on next reboot.
SHA256 (/bsd) = a940ce989d708e5b87a1186ee81bd624066baeabe67b8405b52e4fa2988b565
# dislabel -pm wd0
#size offset fstype [fsize bsize cpg]
a: 353.0M 64 4.2BSD 2048 16384 5624 # /
b: 511.1M 722944swap# none
c: 15280.0M0 unused
d: 444.8M 1769728 4.2BSD 2048 16384 7116 # /tmp
e: 607.7M 2680576 4.2BSD 2048 16384 9685 # /var
f: 1703.0M 3925216 4.2BSD 2048 16384 12958 # /usr
g: 505.8M 7412896 4.2BSD 2048 16384 8060 # /usr/X11R6
h: 1632.9M 8448736 4.2BSD 2048 16384 12958 # /usr/local
i: 1381.2M 11792960 4.2BSD 2048 16384 12958 # /usr/src
j: 5282.4M 14621632 4.2BSD 2048 16384 12958 # /usr/obj
k: 2850.9M 25439936 4.2BSD 2048 16384 12958 # /home
i386 kernel relinking
Question about kernel randomization and relinking...
It seems to take a fair amount of RAM, at least for systems that
are forced to run i386. And I mean real RAM -- swap doesn't seem
to cut it.
I discovered that several machines I was intending on using for
minimal purposes just couldn't complete relinking. So I built a
VM and started playing with the RAM.
Built with 1G RAM, default was a 1.2G swap, worked fine.
Reduced to 256MB RAM, Kernel failed to relink. As with my old
junk.
The magic number seemed to be between 320MB (failed) and 384MB
(worked) of RAM. Ok, fine.
Kernel relinking is important, I get that. Probably time to
start tossing old junk. I get that, too. I'm not complaining
about the forcible retirement of some of my old junk. I'm just
curious why swap didn't "fix" this problem.
But that VM failed at 320MB RAM, even though it had 1.2G of swap,
mostly unused (MOSTLY. Yes, it was going into swap). Is there a
semi-layperson's explanation of this? Or is this a "if you got
to ask, you won't understand" kind of thing?
And here's the relink log from my VM, but the ones from my physical
boxes looked pretty similiar.
$ cat relink.log
(SHA256) /bsd: OK
LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
ld -T ld.script -X --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o ${OBJS}
textdatabss dec hex
0 0 0 0 0
mv newbsd newbsd.gdb
ctfstrip -S -o newbsd newbsd.gdb
strip: there are no sections to be copied!
rm -f bsd.gdb
mv -f newbsd bsd
mv: newbsd: No such file or directory
*** Error 1 in /usr/share/relink/kernel/GENERIC.MP (Makefile:1131 'newbsd')
I also found that a 320MB machine could not build the kernel from scratch.
Nothing used much memory until the ld step, which started using large amounts
of memory and some swap, and errored out the same way:
LD="ld" LDFLAGS="-g" sh makegap.sh 0x gapdummy.o
ld -T ld.script -X --warn-common -nopie -o bsd ${SYSTEM_HEAD} vers.o ${OBJS}
textdatabss dec hex
0 0 0 0 0
mv bsd bsd.gdb
ctfstrip -S -o bsd bsd.gdb
strip: there are no sections to be copied!
Thanks!
Nick.
