Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On 18/08/2016 02:37, Toomas Soome wrote:
> Author: tsoome
> Date: Thu Aug 18 00:37:07 2016
> New Revision: 304321
> URL: https://svnweb.freebsd.org/changeset/base/304321
>
> Log:
> Add SHA512, skein, large blocks support for loader zfs.
>
Hi,
this commit seems to break booting zroot bhyve VMs. Reverting
userboot.so to 304320 on the host system makes it work again.
I start bhyve like this
/usr/share/examples/bhyve/vmrun.sh -d /dev/zvol/zroot/bhyve/poudriere -t
tap8 -c 7 -m 20G poudriere
Launching virtual machine "poudriere" ...
Consoles: userboot
FreeBSD/amd64 User boot, Revision 1.1
(r...@fry.smeets.xyz, Tue Aug 23 16:47:12 CEST 2016)
-
can't load 'kernel'
Type '?' for a list of commands, 'help' for more detailed help.
OK lsdev -v
host devices:
host0: Host filesystem
disk devices:
disk0: Guest drive image
disk0p1: FreeBSD boot512KB
disk0p2: FreeBSD swap2048MB
disk0p3: FreeBSD ZFS 67GB
zfs devices:
OK show
LINES=24
boot_serial=1
console=userboot
currdev=disk0p3:
interpret=OK
loaddev=disk0p3:
prompt=${interpret}
smbios.bios.vendor=BHYVE
twiddle_divisor=1
OK ls
open '/' failed: no such file or directory
OK
Here the zroot setup in the VM
root@poudriere:~ # zpool get -oname,property,value all
NAME PROPERTY VALUE
zroot size 67.5G
zroot capacity 61%
zroot altroot-
zroot health ONLINE
zroot guid 13278473142560840663
zroot version-
zroot bootfs zroot/ROOT/default
zroot delegation on
zroot autoreplaceoff
zroot cachefile -
zroot failmode wait
zroot listsnapshots off
zroot autoexpand off
zroot dedupditto 0
zroot dedupratio 1.00x
zroot free 25.9G
zroot allocated 41.6G
zroot readonly off
zroot comment-
zroot expandsize -
zroot freeing0
zroot fragmentation 67%
zroot leaked 0
zroot feature@async_destroy enabled
zroot feature@empty_bpobjactive
zroot feature@lz4_compress active
zroot feature@multi_vdev_crash_dump enabled
zroot feature@spacemap_histogram active
zroot feature@enabled_txgactive
zroot feature@hole_birth active
zroot feature@extensible_dataset enabled
zroot feature@embedded_data active
zroot feature@bookmarks enabled
zroot feature@filesystem_limits enabled
zroot feature@large_blocks disabled
zroot feature@sha512 disabled
zroot feature@skein disabled
root@poudriere:~ # zfs get all zroot/ROOT/default
NAMEPROPERTY VALUE SOURCE
zroot/ROOT/default type filesystem -
zroot/ROOT/default creation Sun Jul 27 16:32 2014 -
zroot/ROOT/default used 7.12G -
zroot/ROOT/default available 23.8G -
zroot/ROOT/default referenced7.12G -
zroot/ROOT/default compressratio 2.18x -
zroot/ROOT/default mounted yes-
zroot/ROOT/default quota none default
zroot/ROOT/default reservation none default
zroot/ROOT/default recordsize128K default
zroot/ROOT/default mountpoint/ local
zroot/ROOT/default sharenfs offdefault
zroot/ROOT/default checksum on default
zroot/ROOT/default compression lz4
inherited from zroot
zroot/ROOT/default atime off
inherited from zroot
zroot/ROOT/default devices on default
zroot/ROOT/default exec on default
zroot/ROOT/default setuidon default
zroot/ROOT/default readonly offdefault
zroot/ROOT/default jailedoffdefault
zroot/ROOT/default snapdir hidden default
zroot/ROOT/default aclmode discarddefault
zroot/ROOT/default aclinheritrestricted default
zroot/ROOT/default canmount on default
zroot/ROOT/default xattr offtemporary
zroot/ROOT/default copies1 default
zroot/ROOT/default version 5 -
zroot/ROOT/
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 23. aug 2016, at 15:36, Slawa Olhovchenkov wrote: > > On Tue, Aug 23, 2016 at 03:26:04PM +0300, Toomas Soome wrote: > >>> Main trouble (by kib@) is 640KB real mode limit. >>> Separated heap don't soled this. >>> May be solution is early switch to protected mode, boot2? >> >> hm, but boot2 is already btx client and btx is setting up the >> protected mode. Only zfsboot/gpt[zfs]boot memory copy (boot2 >> relocator) is working in segmented real mode, so the gptldr.S and >> zfsldr.S has to deal with memory segments to set boot2 up. Or did >> got you wrong? > > Mat be I am wrong. > May be I am not very clean. > As I understund kib@ restriction caused by 640KB realmode limit. > In case of boot2 start in real mode and read enterly by boot1 -- we > also touch this trouble. > > How to resolve this: > > 1) start entirely; boot2 in protected mode and read all boot2 to > extended memory > > 2) read only part of boot2 (boot2 relocator) by boot1 and switch to > protected mode before reading rest of boot2 by boot2 code. Ah, well, the boot2 by itself is not an issue - it is much smaller and the only related problem was that real mode relocator was using segment size (64k) for copy, but this limit is removed (was pre-requisite for geli support in gptboot and I did port that change to zfsboot as well). The size issue with loader (stage3) is due to fact that loader memory image is located in low memory, base 0xa000 and the upper limit is EBDA and video memory area. Since boot2 is running in protected mode, it can load loader (or kernel) were needed, just that placing loader must be careful… in that sense the current location is almost perfect (with exception about the size limit;) and I would avoid moving it without the very good reason. rgds, toomas ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On Tue, Aug 23, 2016 at 03:26:04PM +0300, Toomas Soome wrote: > > Main trouble (by kib@) is 640KB real mode limit. > > Separated heap don't soled this. > > May be solution is early switch to protected mode, boot2? > > hm, but boot2 is already btx client and btx is setting up the > protected mode. Only zfsboot/gpt[zfs]boot memory copy (boot2 > relocator) is working in segmented real mode, so the gptldr.S and > zfsldr.S has to deal with memory segments to set boot2 up. Or did > got you wrong? Mat be I am wrong. May be I am not very clean. As I understund kib@ restriction caused by 640KB realmode limit. In case of boot2 start in real mode and read enterly by boot1 -- we also touch this trouble. How to resolve this: 1) start entirely; boot2 in protected mode and read all boot2 to extended memory 2) read only part of boot2 (boot2 relocator) by boot1 and switch to protected mode before reading rest of boot2 by boot2 code. ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 23. aug 2016, at 15:16, Slawa Olhovchenkov wrote: > > On Tue, Aug 23, 2016 at 03:00:32PM +0300, Toomas Soome wrote: > >> >>> On 23. aug 2016, at 14:29, Slawa Olhovchenkov wrote: >>> >>> On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote: >>> > On 22. aug 2016, at 17:56, Toomas Soome wrote: > > >> On 22. aug 2016, at 17:19, Warner Losh wrote: >> >> On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: >>> I do suspect the size difference there is partially due to ficl, in >>> illumos (ficl 4): >>> >>> -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a >>> >>> and freebsd (ficl 3): >>> >>> -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a >>> >>> so, there definitely is some space… >> >> Same compiler? Clang bloats the boot code rather substantially, even >> after >> all the flags to tell it to generate smaller code are used. gcc 4.2.x >> built stuff >> was substantially smaller. >> >> There's a 520kb limit enforced in the boot1 for similar reasons. Looks >> like >> the combination of options makes us use just enough extra memory to >> sink the battleship... >> >> Warner >> > > > Actually I only now realized I was comparing apples with oranges… I > forgot the fbsd builds 32bit version in ficl32, this one is 64bit. and > yes the 32bit version is not that big at all:D > > Also, after done some digging, I have found few instances of duplicated > code (we can share sha2 with geli and so if sha512 is already needed, it > will become another “free lunch”). Also, unless I’m mistaken, for some > reason the bzip *compression* is brought in - correct me if I’m wrong, > but afaik only decompression is needed… > > So before going after “useless features”, there are some “hidden” > resources to remove extra fat. > I did some more digging. while ld has —gc-sections to clean up unused bits, to make it effective, the code build does also need -Os -fdata-sections -ffunction-sections. So I did just very simple test by adding those flags to bsd.stand.mk and: first the “default” binaries from /boot: -r-xr-xr-x 1 root wheel 446464 Aug 19 08:46 /boot/zfsloader -rw-r--r-- 1 root wheel 438272 Aug 23 00:30 /boot/zfsloader.b -r-xr-xr-x 1 root wheel 446464 Aug 5 08:37 /boot/zfsloader.old -r--r--r-- 1 root wheel 406568 Aug 19 08:46 /boot/userboot.so (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600) now after adding compile flags -Os -fdata-sections -ffunction-sections: -rw-r--r-- 1 root wheel 389120 Aug 23 10:12 zfsloader -rwxr-xr-x 1 root wheel 378156 Aug 23 10:12 zfsloader.bin -rwxr-xr-x 1 root wheel 437514 Aug 23 10:12 zfsloader.sym -rwxr-xr-x 1 root wheel 375912 Aug 23 10:03 userboot.so and finally test for Andriy with: LOADER_BZIP2_SUPPORT=yes LOADER_FIREWIRE_SUPPORT=yes -rw-r--r-- 1 root wheel 421888 Aug 23 10:22 zfsloader -rwxr-xr-x 1 root wheel 409932 Aug 23 10:22 zfsloader.bin -rwxr-xr-x 1 root wheel 472021 Aug 23 10:22 zfsloader.sym -rwxr-xr-x 1 root wheel 375912 Aug 23 10:22 userboot.so note the userboot.so did not change from those flags. This is just an result from compile, and by adding 3 options to bsd.stand.mk; however, not all Makefiles in loader tree seem to include it, and most importantly, haven’t tested real boot yet;) To conclude, some more work is needed to review the Makefiles, build options used etc, also I don’t know all the background why the compiler options are set as they currently are - were there any related compiler/linker bugs, or any other reasons, also how/if other platforms are affected - for example bsd.stand.mk does set -Os for pc98, but not for others… >>> >>> This is only size on disk, memory consuming still same, IMHO. >> >> Actually this reduction above is entirely due to -Os, the —gc-sections is >> not passed to linker (at least for zfsloader case). I think we will need >> linker script to preserve set_Xcommand_set linker set to use —gc-sections >> with ld. >> >> Loader heap is separate already and does not contribute to this issue. Also, >> I already did test the boot with thinned zfsloader, both with and without >> bzip2 are appearing to work, at least for this quick test anyhow. > > Main trouble (by kib@) is 640KB real mode limit. > Separated heap don't soled this. > May be solution is early switch to protected mode, boot2? hm, but boot2 is already btx client and btx is setting up the protected mode. Only zfsboot/gpt[zfs]boot memory copy (boot2 relocator) is working in segmented real mode, so the gptldr.S and zfsldr.S has to deal with memory
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On Tue, Aug 23, 2016 at 03:00:32PM +0300, Toomas Soome wrote: > > > On 23. aug 2016, at 14:29, Slawa Olhovchenkov wrote: > > > > On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote: > > > >> > >>> On 22. aug 2016, at 17:56, Toomas Soome wrote: > >>> > >>> > On 22. aug 2016, at 17:19, Warner Losh wrote: > > On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: > > I do suspect the size difference there is partially due to ficl, in > > illumos (ficl 4): > > > > -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a > > > > and freebsd (ficl 3): > > > > -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a > > > > so, there definitely is some space… > > Same compiler? Clang bloats the boot code rather substantially, even > after > all the flags to tell it to generate smaller code are used. gcc 4.2.x > built stuff > was substantially smaller. > > There's a 520kb limit enforced in the boot1 for similar reasons. Looks > like > the combination of options makes us use just enough extra memory to > sink the battleship... > > Warner > > >>> > >>> > >>> Actually I only now realized I was comparing apples with oranges… I > >>> forgot the fbsd builds 32bit version in ficl32, this one is 64bit. and > >>> yes the 32bit version is not that big at all:D > >>> > >>> Also, after done some digging, I have found few instances of duplicated > >>> code (we can share sha2 with geli and so if sha512 is already needed, it > >>> will become another “free lunch”). Also, unless I’m mistaken, for some > >>> reason the bzip *compression* is brought in - correct me if I’m wrong, > >>> but afaik only decompression is needed… > >>> > >>> So before going after “useless features”, there are some “hidden” > >>> resources to remove extra fat. > >>> > >> > >> I did some more digging. while ld has —gc-sections to clean up unused > >> bits, to make it effective, the code build does also need -Os > >> -fdata-sections -ffunction-sections. > >> So I did just very simple test by adding those flags to bsd.stand.mk and: > >> > >> first the “default” binaries from /boot: > >> -r-xr-xr-x 1 root wheel 446464 Aug 19 08:46 /boot/zfsloader > >> -rw-r--r-- 1 root wheel 438272 Aug 23 00:30 /boot/zfsloader.b > >> -r-xr-xr-x 1 root wheel 446464 Aug 5 08:37 /boot/zfsloader.old > >> -r--r--r-- 1 root wheel 406568 Aug 19 08:46 /boot/userboot.so > >> > >> (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600) > >> > >> now after adding compile flags -Os -fdata-sections -ffunction-sections: > >> > >> -rw-r--r-- 1 root wheel 389120 Aug 23 10:12 zfsloader > >> -rwxr-xr-x 1 root wheel 378156 Aug 23 10:12 zfsloader.bin > >> -rwxr-xr-x 1 root wheel 437514 Aug 23 10:12 zfsloader.sym > >> -rwxr-xr-x 1 root wheel 375912 Aug 23 10:03 userboot.so > >> > >> and finally test for Andriy with: > >> LOADER_BZIP2_SUPPORT=yes > >> LOADER_FIREWIRE_SUPPORT=yes > >> > >> -rw-r--r-- 1 root wheel 421888 Aug 23 10:22 zfsloader > >> -rwxr-xr-x 1 root wheel 409932 Aug 23 10:22 zfsloader.bin > >> -rwxr-xr-x 1 root wheel 472021 Aug 23 10:22 zfsloader.sym > >> -rwxr-xr-x 1 root wheel 375912 Aug 23 10:22 userboot.so > >> > >> note the userboot.so did not change from those flags. > >> > >> This is just an result from compile, and by adding 3 options to > >> bsd.stand.mk; however, not all Makefiles in loader tree seem to include > >> it, and most importantly, haven’t tested real boot yet;) > >> > >> To conclude, some more work is needed to review the Makefiles, build > >> options used etc, also I don’t know all the background why the compiler > >> options are set as they currently are - were there any related > >> compiler/linker bugs, or any other reasons, also how/if other platforms > >> are affected - for example bsd.stand.mk does set -Os for pc98, but not for > >> others… > > > > This is only size on disk, memory consuming still same, IMHO. > > Actually this reduction above is entirely due to -Os, the —gc-sections is not > passed to linker (at least for zfsloader case). I think we will need linker > script to preserve set_Xcommand_set linker set to use —gc-sections with ld. > > Loader heap is separate already and does not contribute to this issue. Also, > I already did test the boot with thinned zfsloader, both with and without > bzip2 are appearing to work, at least for this quick test anyhow. Main trouble (by kib@) is 640KB real mode limit. Separated heap don't soled this. May be solution is early switch to protected mode, boot2? ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 23. aug 2016, at 14:29, Slawa Olhovchenkov wrote: > > On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote: > >> >>> On 22. aug 2016, at 17:56, Toomas Soome wrote: >>> >>> On 22. aug 2016, at 17:19, Warner Losh wrote: On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: > I do suspect the size difference there is partially due to ficl, in > illumos (ficl 4): > > -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a > > and freebsd (ficl 3): > > -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a > > so, there definitely is some space… Same compiler? Clang bloats the boot code rather substantially, even after all the flags to tell it to generate smaller code are used. gcc 4.2.x built stuff was substantially smaller. There's a 520kb limit enforced in the boot1 for similar reasons. Looks like the combination of options makes us use just enough extra memory to sink the battleship... Warner >>> >>> >>> Actually I only now realized I was comparing apples with oranges… I forgot >>> the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the >>> 32bit version is not that big at all:D >>> >>> Also, after done some digging, I have found few instances of duplicated >>> code (we can share sha2 with geli and so if sha512 is already needed, it >>> will become another “free lunch”). Also, unless I’m mistaken, for some >>> reason the bzip *compression* is brought in - correct me if I’m wrong, but >>> afaik only decompression is needed… >>> >>> So before going after “useless features”, there are some “hidden” resources >>> to remove extra fat. >>> >> >> I did some more digging. while ld has —gc-sections to clean up unused bits, >> to make it effective, the code build does also need -Os -fdata-sections >> -ffunction-sections. >> So I did just very simple test by adding those flags to bsd.stand.mk and: >> >> first the “default” binaries from /boot: >> -r-xr-xr-x 1 root wheel 446464 Aug 19 08:46 /boot/zfsloader >> -rw-r--r-- 1 root wheel 438272 Aug 23 00:30 /boot/zfsloader.b >> -r-xr-xr-x 1 root wheel 446464 Aug 5 08:37 /boot/zfsloader.old >> -r--r--r-- 1 root wheel 406568 Aug 19 08:46 /boot/userboot.so >> >> (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600) >> >> now after adding compile flags -Os -fdata-sections -ffunction-sections: >> >> -rw-r--r-- 1 root wheel 389120 Aug 23 10:12 zfsloader >> -rwxr-xr-x 1 root wheel 378156 Aug 23 10:12 zfsloader.bin >> -rwxr-xr-x 1 root wheel 437514 Aug 23 10:12 zfsloader.sym >> -rwxr-xr-x 1 root wheel 375912 Aug 23 10:03 userboot.so >> >> and finally test for Andriy with: >> LOADER_BZIP2_SUPPORT=yes >> LOADER_FIREWIRE_SUPPORT=yes >> >> -rw-r--r-- 1 root wheel 421888 Aug 23 10:22 zfsloader >> -rwxr-xr-x 1 root wheel 409932 Aug 23 10:22 zfsloader.bin >> -rwxr-xr-x 1 root wheel 472021 Aug 23 10:22 zfsloader.sym >> -rwxr-xr-x 1 root wheel 375912 Aug 23 10:22 userboot.so >> >> note the userboot.so did not change from those flags. >> >> This is just an result from compile, and by adding 3 options to >> bsd.stand.mk; however, not all Makefiles in loader tree seem to include it, >> and most importantly, haven’t tested real boot yet;) >> >> To conclude, some more work is needed to review the Makefiles, build options >> used etc, also I don’t know all the background why the compiler options are >> set as they currently are - were there any related compiler/linker bugs, or >> any other reasons, also how/if other platforms are affected - for example >> bsd.stand.mk does set -Os for pc98, but not for others… > > This is only size on disk, memory consuming still same, IMHO. Actually this reduction above is entirely due to -Os, the —gc-sections is not passed to linker (at least for zfsloader case). I think we will need linker script to preserve set_Xcommand_set linker set to use —gc-sections with ld. Loader heap is separate already and does not contribute to this issue. Also, I already did test the boot with thinned zfsloader, both with and without bzip2 are appearing to work, at least for this quick test anyhow. rgds, toomas ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote: > > > On 22. aug 2016, at 17:56, Toomas Soome wrote: > > > > > >> On 22. aug 2016, at 17:19, Warner Losh wrote: > >> > >> On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: > >>> I do suspect the size difference there is partially due to ficl, in > >>> illumos (ficl 4): > >>> > >>> -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a > >>> > >>> and freebsd (ficl 3): > >>> > >>> -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a > >>> > >>> so, there definitely is some space… > >> > >> Same compiler? Clang bloats the boot code rather substantially, even after > >> all the flags to tell it to generate smaller code are used. gcc 4.2.x > >> built stuff > >> was substantially smaller. > >> > >> There's a 520kb limit enforced in the boot1 for similar reasons. Looks like > >> the combination of options makes us use just enough extra memory to > >> sink the battleship... > >> > >> Warner > >> > > > > > > Actually I only now realized I was comparing apples with oranges… I forgot > > the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the > > 32bit version is not that big at all:D > > > > Also, after done some digging, I have found few instances of duplicated > > code (we can share sha2 with geli and so if sha512 is already needed, it > > will become another “free lunch”). Also, unless I’m mistaken, for some > > reason the bzip *compression* is brought in - correct me if I’m wrong, but > > afaik only decompression is needed… > > > > So before going after “useless features”, there are some “hidden” resources > > to remove extra fat. > > > > I did some more digging. while ld has —gc-sections to clean up unused bits, > to make it effective, the code build does also need -Os -fdata-sections > -ffunction-sections. > So I did just very simple test by adding those flags to bsd.stand.mk and: > > first the “default” binaries from /boot: > -r-xr-xr-x 1 root wheel 446464 Aug 19 08:46 /boot/zfsloader > -rw-r--r-- 1 root wheel 438272 Aug 23 00:30 /boot/zfsloader.b > -r-xr-xr-x 1 root wheel 446464 Aug 5 08:37 /boot/zfsloader.old > -r--r--r-- 1 root wheel 406568 Aug 19 08:46 /boot/userboot.so > > (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600) > > now after adding compile flags -Os -fdata-sections -ffunction-sections: > > -rw-r--r-- 1 root wheel 389120 Aug 23 10:12 zfsloader > -rwxr-xr-x 1 root wheel 378156 Aug 23 10:12 zfsloader.bin > -rwxr-xr-x 1 root wheel 437514 Aug 23 10:12 zfsloader.sym > -rwxr-xr-x 1 root wheel 375912 Aug 23 10:03 userboot.so > > and finally test for Andriy with: > LOADER_BZIP2_SUPPORT=yes > LOADER_FIREWIRE_SUPPORT=yes > > -rw-r--r-- 1 root wheel 421888 Aug 23 10:22 zfsloader > -rwxr-xr-x 1 root wheel 409932 Aug 23 10:22 zfsloader.bin > -rwxr-xr-x 1 root wheel 472021 Aug 23 10:22 zfsloader.sym > -rwxr-xr-x 1 root wheel 375912 Aug 23 10:22 userboot.so > > note the userboot.so did not change from those flags. > > This is just an result from compile, and by adding 3 options to bsd.stand.mk; > however, not all Makefiles in loader tree seem to include it, and most > importantly, haven’t tested real boot yet;) > > To conclude, some more work is needed to review the Makefiles, build options > used etc, also I don’t know all the background why the compiler options are > set as they currently are - were there any related compiler/linker bugs, or > any other reasons, also how/if other platforms are affected - for example > bsd.stand.mk does set -Os for pc98, but not for others… This is only size on disk, memory consuming still same, IMHO. ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 23. aug 2016, at 11:22, Andriy Gapon wrote: > > On 22/08/2016 17:56, Toomas Soome wrote: >> Actually I only now realized I was comparing apples with oranges… I forgot >> the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the 32bit >> version is not that big at all:D >> >> Also, after done some digging, I have found few instances of duplicated code >> (we can share sha2 with geli and so if sha512 is already needed, it will >> become another “free lunch”). Also, unless I’m mistaken, for some reason the >> bzip *compression* is brought in - correct me if I’m wrong, but afaik only >> decompression is needed… >> >> So before going after “useless features”, there are some “hidden” resources >> to remove extra fat. > > I certainly agree with this and those things would be good to do. > But if we do not change the trend then sooner or later we will run out of > things > that we can optimize. But it's also possible that the current limitations > will > be a history by then. > > -- > Andriy Gapon Yes, also from my illumos work, even building framebuffer based GFX interface haven’t added too much extra code, but it is important to recognize the limits, and this issue did provide really good lesson about it;) Right now there are two tasks to finalize - complete the review/test/fix for proper compiler options, and to understand the actual limits regarding userboot module - for me it is absolutely unknown area. rgds, toomas ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On 22/08/2016 17:56, Toomas Soome wrote: > Actually I only now realized I was comparing apples with oranges… I forgot > the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the 32bit > version is not that big at all:D > > Also, after done some digging, I have found few instances of duplicated code > (we can share sha2 with geli and so if sha512 is already needed, it will > become another “free lunch”). Also, unless I’m mistaken, for some reason the > bzip *compression* is brought in - correct me if I’m wrong, but afaik only > decompression is needed… > > So before going after “useless features”, there are some “hidden” resources > to remove extra fat. I certainly agree with this and those things would be good to do. But if we do not change the trend then sooner or later we will run out of things that we can optimize. But it's also possible that the current limitations will be a history by then. -- Andriy Gapon ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 22. aug 2016, at 17:56, Toomas Soome wrote: > > >> On 22. aug 2016, at 17:19, Warner Losh wrote: >> >> On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: >>> I do suspect the size difference there is partially due to ficl, in illumos >>> (ficl 4): >>> >>> -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a >>> >>> and freebsd (ficl 3): >>> >>> -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a >>> >>> so, there definitely is some space… >> >> Same compiler? Clang bloats the boot code rather substantially, even after >> all the flags to tell it to generate smaller code are used. gcc 4.2.x >> built stuff >> was substantially smaller. >> >> There's a 520kb limit enforced in the boot1 for similar reasons. Looks like >> the combination of options makes us use just enough extra memory to >> sink the battleship... >> >> Warner >> > > > Actually I only now realized I was comparing apples with oranges… I forgot > the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the 32bit > version is not that big at all:D > > Also, after done some digging, I have found few instances of duplicated code > (we can share sha2 with geli and so if sha512 is already needed, it will > become another “free lunch”). Also, unless I’m mistaken, for some reason the > bzip *compression* is brought in - correct me if I’m wrong, but afaik only > decompression is needed… > > So before going after “useless features”, there are some “hidden” resources > to remove extra fat. > I did some more digging. while ld has —gc-sections to clean up unused bits, to make it effective, the code build does also need -Os -fdata-sections -ffunction-sections. So I did just very simple test by adding those flags to bsd.stand.mk and: first the “default” binaries from /boot: -r-xr-xr-x 1 root wheel 446464 Aug 19 08:46 /boot/zfsloader -rw-r--r-- 1 root wheel 438272 Aug 23 00:30 /boot/zfsloader.b -r-xr-xr-x 1 root wheel 446464 Aug 5 08:37 /boot/zfsloader.old -r--r--r-- 1 root wheel 406568 Aug 19 08:46 /boot/userboot.so (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600) now after adding compile flags -Os -fdata-sections -ffunction-sections: -rw-r--r-- 1 root wheel 389120 Aug 23 10:12 zfsloader -rwxr-xr-x 1 root wheel 378156 Aug 23 10:12 zfsloader.bin -rwxr-xr-x 1 root wheel 437514 Aug 23 10:12 zfsloader.sym -rwxr-xr-x 1 root wheel 375912 Aug 23 10:03 userboot.so and finally test for Andriy with: LOADER_BZIP2_SUPPORT=yes LOADER_FIREWIRE_SUPPORT=yes -rw-r--r-- 1 root wheel 421888 Aug 23 10:22 zfsloader -rwxr-xr-x 1 root wheel 409932 Aug 23 10:22 zfsloader.bin -rwxr-xr-x 1 root wheel 472021 Aug 23 10:22 zfsloader.sym -rwxr-xr-x 1 root wheel 375912 Aug 23 10:22 userboot.so note the userboot.so did not change from those flags. This is just an result from compile, and by adding 3 options to bsd.stand.mk; however, not all Makefiles in loader tree seem to include it, and most importantly, haven’t tested real boot yet;) To conclude, some more work is needed to review the Makefiles, build options used etc, also I don’t know all the background why the compiler options are set as they currently are - were there any related compiler/linker bugs, or any other reasons, also how/if other platforms are affected - for example bsd.stand.mk does set -Os for pc98, but not for others… rgds, toomas ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 22. aug 2016, at 17:19, Warner Losh wrote: > > On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: >> I do suspect the size difference there is partially due to ficl, in illumos >> (ficl 4): >> >> -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a >> >> and freebsd (ficl 3): >> >> -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a >> >> so, there definitely is some space… > > Same compiler? Clang bloats the boot code rather substantially, even after > all the flags to tell it to generate smaller code are used. gcc 4.2.x > built stuff > was substantially smaller. > > There's a 520kb limit enforced in the boot1 for similar reasons. Looks like > the combination of options makes us use just enough extra memory to > sink the battleship... > > Warner > Actually I only now realized I was comparing apples with oranges… I forgot the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the 32bit version is not that big at all:D Also, after done some digging, I have found few instances of duplicated code (we can share sha2 with geli and so if sha512 is already needed, it will become another “free lunch”). Also, unless I’m mistaken, for some reason the bzip *compression* is brought in - correct me if I’m wrong, but afaik only decompression is needed… So before going after “useless features”, there are some “hidden” resources to remove extra fat. rgds, toomas ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome wrote: > I do suspect the size difference there is partially due to ficl, in illumos > (ficl 4): > > -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a > > and freebsd (ficl 3): > > -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a > > so, there definitely is some space… Same compiler? Clang bloats the boot code rather substantially, even after all the flags to tell it to generate smaller code are used. gcc 4.2.x built stuff was substantially smaller. There's a 520kb limit enforced in the boot1 for similar reasons. Looks like the combination of options makes us use just enough extra memory to sink the battleship... Warner ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
> On 22. aug 2016, at 11:09, Andriy Gapon wrote: > > On 22/08/2016 10:20, Andriy Gapon wrote: >> This commit breaks boot process for me and in a quite weird way. >> I don't have a serial console, so a couple of screenshots. >> This is what happens with this change: >> https://people.freebsd.org/~avg/boot-fail-1024x768.jpg >> This is what I have with the previous loader: >> https://people.freebsd.org/~avg/boot-success-1024x768.jpg >> >> As you can see somehow the HDD gets misdetected as a floppy, BIOS disk ID is >> 0x0 >> as opposed to 0x80. Also, the disk size is incorrect too. Additionally the >> firewire is not detected. >> >> I suspect that the problem may have to do with the increased loader size. >> I must add that I have these in src.conf: >> LOADER_BZIP2_SUPPORT=yes >> LOADER_FIREWIRE_SUPPORT=yes > > Removing both of those options allows the boot to succeed. > Which sort of, maybe confirms the hypothesis. > yep, just enabling firewire does not blow it. > Also, as extra data points, this is how SMAP is reported by a good zfsloader: > https://people.freebsd.org/~avg/boot-smap-1024x768.jpg > And it seems to be corrupted when using the bad zfsloader: > https://people.freebsd.org/~avg/boot-smap-corrupt-1024x768.jpg > > Base memory size is 634880 (almost enough for everyone). > Extended memory is ~ 3.5GB and the high memory is 64MB at the top of it. > > File size of the loader that does not work is 483328 bytes. > $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin >textdata bss dec hex filename > 438000 26416 130896 595312 0x91570 > /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin > > File size of the loader that works is 450560 bytes. > $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin >textdata bss dec hex filename > 410920 23304 50636 484860 0x765fc > /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin > > So, it seems that there is a practical limit on a loader size for real-world > x86 > BIOS-based systems. We are very close to the limit with the default ZFS > loader > and we cross that limit if additional loader features are enabled. > > My opinion is that we should stop cramming all possible ZFS features into the > loader. Instead we should admit that the boot pool, or at least boot > filesystem, must have certain limitations on features that it can use. Then > there is no need to add support for those features to the loader. Well, it is arguable if its more important to have bzip in loader or support for checksum algorithms, especially as zfs does compression, but lets not get onto this slippery ground;) but, yes, the root cause of the issue is that if additional [relatively large] code is included by src.conf setup, we will hit the ceiling of <1MB memory area, as client base (0xa000) + code + data are all occupying sequential memory and we are reaching EBDA area in low memory. And indeed, it does mean more conservative approach about what is included and how. btw, just to compare, zfsloader with ficl 4 + zfs additionally with gzip (as gzip is needed for for other reasons anyhow, so its “free lunch”) in my illumos build is: -r--r--r-- 1 root sys 434176 aug 21 16:04 /boot/zfsloader and the current head + firewire in freebsd is: -rw-r--r-- 1 root wheel 454656 Aug 22 11:54 /boot/zfsloader I do suspect the size difference there is partially due to ficl, in illumos (ficl 4): -rw-r--r-- 1 tsoome staff 132508 aug 22 09:18 libficl.a and freebsd (ficl 3): -rw-r--r-- 1 root wheel 213748 Aug 19 01:57 libficl.a so, there definitely is some space… rgds, toomas > > Personally, I would prefer that this commit is backed out unless it can be > strongly justified. Unless we can quickly find a way to run the loader at a > different, less restricted memory location. > >> My memory of loader's memory placement and layout has faded, so I can't >> elaborate further on my guess. >> >> Also, there is this report: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212038 >> That problem could have a different cause. It should be easier to analyze as >> the it happens with bhyveload, i.e., in userland. > > > -- > Andriy Gapon ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On 22/08/2016 10:20, Andriy Gapon wrote: > This commit breaks boot process for me and in a quite weird way. > I don't have a serial console, so a couple of screenshots. > This is what happens with this change: > https://people.freebsd.org/~avg/boot-fail-1024x768.jpg > This is what I have with the previous loader: > https://people.freebsd.org/~avg/boot-success-1024x768.jpg > > As you can see somehow the HDD gets misdetected as a floppy, BIOS disk ID is > 0x0 > as opposed to 0x80. Also, the disk size is incorrect too. Additionally the > firewire is not detected. > > I suspect that the problem may have to do with the increased loader size. > I must add that I have these in src.conf: > LOADER_BZIP2_SUPPORT=yes > LOADER_FIREWIRE_SUPPORT=yes Removing both of those options allows the boot to succeed. Which sort of, maybe confirms the hypothesis. Also, as extra data points, this is how SMAP is reported by a good zfsloader: https://people.freebsd.org/~avg/boot-smap-1024x768.jpg And it seems to be corrupted when using the bad zfsloader: https://people.freebsd.org/~avg/boot-smap-corrupt-1024x768.jpg Base memory size is 634880 (almost enough for everyone). Extended memory is ~ 3.5GB and the high memory is 64MB at the top of it. File size of the loader that does not work is 483328 bytes. $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin textdata bss dec hex filename 438000 26416 130896 595312 0x91570 /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin File size of the loader that works is 450560 bytes. $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin textdata bss dec hex filename 410920 23304 50636 484860 0x765fc /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin So, it seems that there is a practical limit on a loader size for real-world x86 BIOS-based systems. We are very close to the limit with the default ZFS loader and we cross that limit if additional loader features are enabled. My opinion is that we should stop cramming all possible ZFS features into the loader. Instead we should admit that the boot pool, or at least boot filesystem, must have certain limitations on features that it can use. Then there is no need to add support for those features to the loader. Personally, I would prefer that this commit is backed out unless it can be strongly justified. Unless we can quickly find a way to run the loader at a different, less restricted memory location. > My memory of loader's memory placement and layout has faded, so I can't > elaborate further on my guess. > > Also, there is this report: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212038 > That problem could have a different cause. It should be easier to analyze as > the it happens with bhyveload, i.e., in userland. -- Andriy Gapon ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z
On 18/08/2016 03:37, Toomas Soome wrote: > Author: tsoome > Date: Thu Aug 18 00:37:07 2016 > New Revision: 304321 > URL: https://svnweb.freebsd.org/changeset/base/304321 > > Log: > Add SHA512, skein, large blocks support for loader zfs. > > Updated sha512 from illumos. > Using skein from freebsd crypto tree. > Since loader itself is using 64MB memory for heap, updated zfsboot to > use same, and this also allows to support zfs large blocks. > > Note, adding additional features does increate zfsboot code, therefore > this update does increase zfsboot code to 128k, also I have ported gptldr.S > update to zfsldr.S to support 64k+ code. This commit breaks boot process for me and in a quite weird way. I don't have a serial console, so a couple of screenshots. This is what happens with this change: https://people.freebsd.org/~avg/boot-fail-1024x768.jpg This is what I have with the previous loader: https://people.freebsd.org/~avg/boot-success-1024x768.jpg As you can see somehow the HDD gets misdetected as a floppy, BIOS disk ID is 0x0 as opposed to 0x80. Also, the disk size is incorrect too. Additionally the firewire is not detected. I suspect that the problem may have to do with the increased loader size. I must add that I have these in src.conf: LOADER_BZIP2_SUPPORT=yes LOADER_FIREWIRE_SUPPORT=yes My memory of loader's memory placement and layout has faded, so I can't elaborate further on my guess. Also, there is this report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212038 That problem could have a different cause. It should be easier to analyze as the it happens with bhyveload, i.e., in userland. -- Andriy Gapon ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
