WITHOUT_CAPSICUM|CASPER option ignored: it is no longer supported
In exercising the build options on 14.0-BETA4, WITHOUT_CAPSICUM and WITHOUT_CASPER appear to be in an ambiguous state: They are present but ignored. Fortunately src.conf(5) now reports "This option has no effect." Will these be removed prior to the final release? Are they staying to be reimplemented? Thank you! Michael
Re: something magic about the size of a ports tree
On 2023-10-03 12:24, Dag-Erling Smørgrav wrote: Matthias Apitz writes: I have on my poudriere build host a ports tree and wanted to move it to the host where the resulting packages are installed: root@jet:/usr/local/poudriere/ports # du -sh ports20230806 397Mports20230806 root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806 root@jet:/usr/local/poudriere/ports # ls -lh p.tar -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar already the size of the tar file is somewhat magic; but if you un-tar it on the other host I will get: [guru@c720-1400094 ~]$ ls -lh p.tar -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar [guru@c720-1400094 ~]$ tar xf p.tar [guru@c720-1400094 ~]$ du -sh ports20230806 1,2Gports20230806 How this is possible? Most files in the ports tree are very small. On disk, each file gets rounded up to the nearest multiple of the filesystem block size, which could be as small as 512 bytes or as large as 8 kB (or even more in pathological cases). In a tarball, they get rounded up to the nearest multiple of 512 bytes plus an additional 512 bytes per file for metadata. For instance, your average distinfo file (of which there are 30k in the ports tree) is only 200-250 bytes long, but it occupies 512 bytes on an FFS filesystem, 1 kB in a tarball, and 4 kB on a typical ZFS filesystem. As an interesting side note to this, if ZFS is able to compress the file to under 112 bytes, ZFS will not allocate a sector, but instead store the file in an "embedded blockpointer", basically using the space it would normally store the LBAs and checksum of the file, to store the actual file data, resulting in a file that appears to use 0 bytes of space, because it entirely fits in the indirect block that would have pointed to the block itself. Note that if the target system is FreeBSD 14 or newer, you can simply mount the tarball (`sudo mount -rt tarfs p.tar /usr/ports`). DES -- Allan Jude
Re: something magic about the size of a ports tree
Warner Losh writes: > Do we support any compression on top of that? Has support for > poudriere been added for it? Yes (zstd) and no. DES -- Dag-Erling Smørgrav - d...@freebsd.org
Re: something magic about the size of a ports tree
On Tue, Oct 3, 2023, 10:24 AM Dag-Erling Smørgrav wrote: > Matthias Apitz writes: > > I have on my poudriere build host a ports tree and wanted to move it to > > the host where the resulting packages are installed: > > > > root@jet:/usr/local/poudriere/ports # du -sh ports20230806 > > 397Mports20230806 > > root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806 > > root@jet:/usr/local/poudriere/ports # ls -lh p.tar > > -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar > > > > already the size of the tar file is somewhat magic; but if you un-tar it > > on the other host I will get: > > > > [guru@c720-1400094 ~]$ ls -lh p.tar > > -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar > > [guru@c720-1400094 ~]$ tar xf p.tar > > [guru@c720-1400094 ~]$ du -sh ports20230806 > > 1,2G ports20230806 > > > > How this is possible? > > Most files in the ports tree are very small. On disk, each file gets > rounded up to the nearest multiple of the filesystem block size, which > could be as small as 512 bytes or as large as 8 kB (or even more in > pathological cases). In a tarball, they get rounded up to the nearest > multiple of 512 bytes plus an additional 512 bytes per file for > metadata. > > For instance, your average distinfo file (of which there are 30k in the > ports tree) is only 200-250 bytes long, but it occupies 512 bytes on an > FFS filesystem, 1 kB in a tarball, and 4 kB on a typical ZFS filesystem. > > Note that if the target system is FreeBSD 14 or newer, you can simply > mount the tarball (`sudo mount -rt tarfs p.tar /usr/ports`). > Do we support any compression on top of that? Has support for poudriere been added for it? Aldo I want a pony I'm mostly curious... I have no immediate plans here (though aligning with the boot loader and supporting this on a block device to support rootfs would be cool). Maybe some or all of these wishes would make good GSOC projects? Warner DES > -- > Dag-Erling Smørgrav - d...@freebsd.org > >
Re: something magic about the size of a ports tree
> On 3. Oct 2023, at 18:27, Matthias Apitz wrote: > > El día martes, octubre 03, 2023 a las 06:14:23p. m. +0200, Olivier Certner > escribió: > >> Hi Matthias, >> >> Some ZFS dataset with zstd compression on jet, and no compression on >> c720-1400094? >> > > Yes, on jet it is ZFS: > > root@jet:/usr/local/poudriere/ports # mount | grep ports2023 > poudriere/poudriere/ports/ports20230806 on > /usr/local/poudriere/ports/ports20230806 (zfs, local, noatime, nfsv4acls) > > on c720-1400094 it is only plain UFS. > Try du -hA file Also, to experience the difference, try: dd if=/dev/zero of=tempfile bs=1m count=10 and compare the results of ls, du -h, du -hA on the different filesystems. zfs get all | grep compr can also be quite enlightening. Cheers >matthias > > -- > Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub >
Re: something magic about the size of a ports tree
With ZFS, you might be using transparent compression. "du -sh" will show you a file's compressed size. But "ls -lh" will show you the logical size. That's probably why the tarball looked so much bigger than the ports tree on the first system. If you do "du -sh" on the tarball, I bet you'll see a much smaller number. On Tue, Oct 3, 2023 at 9:27 AM Matthias Apitz wrote: > > El día martes, octubre 03, 2023 a las 06:14:23p. m. +0200, Olivier Certner > escribió: > > > Hi Matthias, > > > > Some ZFS dataset with zstd compression on jet, and no compression on > > c720-1400094? > > > > Yes, on jet it is ZFS: > > root@jet:/usr/local/poudriere/ports # mount | grep ports2023 > poudriere/poudriere/ports/ports20230806 on > /usr/local/poudriere/ports/ports20230806 (zfs, local, noatime, nfsv4acls) > > on c720-1400094 it is only plain UFS. > > matthias > > -- > Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub >
Re: something magic about the size of a ports tree
El día martes, octubre 03, 2023 a las 06:14:23p. m. +0200, Olivier Certner escribió: > Hi Matthias, > > Some ZFS dataset with zstd compression on jet, and no compression on > c720-1400094? > Yes, on jet it is ZFS: root@jet:/usr/local/poudriere/ports # mount | grep ports2023 poudriere/poudriere/ports/ports20230806 on /usr/local/poudriere/ports/ports20230806 (zfs, local, noatime, nfsv4acls) on c720-1400094 it is only plain UFS. matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub
Re: something magic about the size of a ports tree
Matthias Apitz writes: > I have on my poudriere build host a ports tree and wanted to move it to > the host where the resulting packages are installed: > > root@jet:/usr/local/poudriere/ports # du -sh ports20230806 > 397Mports20230806 > root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806 > root@jet:/usr/local/poudriere/ports # ls -lh p.tar > -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar > > already the size of the tar file is somewhat magic; but if you un-tar it > on the other host I will get: > > [guru@c720-1400094 ~]$ ls -lh p.tar > -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar > [guru@c720-1400094 ~]$ tar xf p.tar > [guru@c720-1400094 ~]$ du -sh ports20230806 > 1,2G ports20230806 > > How this is possible? Most files in the ports tree are very small. On disk, each file gets rounded up to the nearest multiple of the filesystem block size, which could be as small as 512 bytes or as large as 8 kB (or even more in pathological cases). In a tarball, they get rounded up to the nearest multiple of 512 bytes plus an additional 512 bytes per file for metadata. For instance, your average distinfo file (of which there are 30k in the ports tree) is only 200-250 bytes long, but it occupies 512 bytes on an FFS filesystem, 1 kB in a tarball, and 4 kB on a typical ZFS filesystem. Note that if the target system is FreeBSD 14 or newer, you can simply mount the tarball (`sudo mount -rt tarfs p.tar /usr/ports`). DES -- Dag-Erling Smørgrav - d...@freebsd.org
Re: something magic about the size of a ports tree
Hi Matthias, Some ZFS dataset with zstd compression on jet, and no compression on c720-1400094? -- Olivier Certner
something magic about the size of a ports tree
I have on my poudriere build host a ports tree and wanted to move it to the host where the resulting packages are installed: root@jet:/usr/local/poudriere/ports # du -sh ports20230806 397Mports20230806 root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806 root@jet:/usr/local/poudriere/ports # ls -lh p.tar -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar already the size of the tar file is somewhat magic; but if you un-tar it on the other host I will get: [guru@c720-1400094 ~]$ ls -lh p.tar -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar [guru@c720-1400094 ~]$ tar xf p.tar [guru@c720-1400094 ~]$ du -sh ports20230806 1,2Gports20230806 How this is possible? matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub
base-krb5 issues (segfaults when adding principals in openssl)
Hi,
has someone else issues with krb5 on -current when adding principals?
With -current as of 2023-09-11 I get a segfault in openssl:
---snip---
Reading symbols from /usr/bin/kadmin...
Reading symbols from /usr/lib/debug//usr/bin/kadmin.debug...
[New LWP 270171]
btCore was generated by `kadmin -l'.
Program terminated with signal SIGSEGV, Segmentation fault.
Address not mapped to object.
#0 0x in ?? ()
(gdb) bt
#0 0x in ?? ()
#1 0x0e118da145f8 in ARCFOUR_string_to_key (context=0x44f9fba1a000,
enctype=KRB5_ENCTYPE_ARCFOUR_HMAC_MD5, password=..., salt=...,
opaque=..., key=0x44f9fba211d8)
at /space/system/usr_src/crypto/heimdal/lib/krb5/salt-arcfour.c:84
#2 0x0e118da156e9 in krb5_string_to_key_data_salt_opaque
(enctype=KRB5_ENCTYPE_ARCFOUR_HMAC_MD5, salt=..., opaque=...,
context=, password=...,
key=) at
/space/system/usr_src/crypto/heimdal/lib/krb5/salt.c:201
#3 krb5_string_to_key_data_salt (context=0x44f9fba1a000,
enctype=KRB5_ENCTYPE_ARCFOUR_HMAC_MD5, password=..., salt=...,
key=0x44f9fba211d8)
at /space/system/usr_src/crypto/heimdal/lib/krb5/salt.c:173
#4 0x0e118da158cb in krb5_string_to_key_salt
(context=0x44f9fba4bc60, context@entry=0x44f9fba1a000,
enctype=-1980854121, password=0x0,
password@entry=0xe1189ee9510 "1kad$uwi6!", salt=..., key=0x5) at
/space/system/usr_src/crypto/heimdal/lib/krb5/salt.c:225
#5 0x0e118ba75423 in hdb_generate_key_set_password
(context=0x44f9fba1a000, principal=,
password=password@entry=0xe1189ee9510 "1kad$uwi6!",
keys=keys@entry=0xe1189ee9210,
num_keys=num_keys@entry=0xe1189ee9208) at
/space/system/usr_src/crypto/heimdal/lib/hdb/keys.c:381
#6 0x0e118ca91c9a in _kadm5_set_keys
(context=context@entry=0x44f9fba1a140, ent=ent@entry=0xe1189ee9258,
password=0x1 ,
password@entry=0xe1189ee9510 "1kad$uwi6!") at
/space/system/usr_src/crypto/heimdal/lib/kadm5/set_keys.c:51
#7 0x0e118ca8caac in kadm5_s_create_principal
(server_handle=0x44f9fba1a140, princ=, mask=out>, password=0xe1189ee9510 "1kad$uwi6!")
at /space/system/usr_src/crypto/heimdal/lib/kadm5/create_s.c:172
#8 0x0e0969e1a57b in add_one_principal (name=,
rand_key=0, rand_password=0, use_defaults=0, password=0xe1189ee9510
"1kad$uwi6!", key_data=0x0,
max_ticket_life=, max_renewable_life=,
attributes=0x0, expiration=, pw_expiration=0x0)
at /space/system/usr_src/crypto/heimdal/kadmin/ank.c:141
#9 add_new_key (opt=opt@entry=0xe1189ee9960, argc=argc@entry=1,
argv=0x44f9fba49238, argv@entry=0x44f9fba49230) at
/space/system/usr_src/crypto/heimdal/kadmin/ank.c:243
#10 0x0e0969e1e124 in add_wrap (argc=,
argv=0x44f9fba49230) at kadmin-commands.c:210
#11 0x0e0969e23945 in sl_command (cmds=, argc=2,
argv=0x44f9fba49230) at
/space/system/usr_src/crypto/heimdal/lib/sl/sl.c:209
#12 sl_command_loop (cmds=cmds@entry=0xe0969e282a0 ,
prompt=prompt@entry=0xe0969e15cca "kadmin> ", data=)
at /space/system/usr_src/crypto/heimdal/lib/sl/sl.c:328
#13 0x0e0969e1d876 in main (argc=, argv=out>) at /space/system/usr_src/crypto/heimdal/kadmin/kadmin.c:275
(gdb) up 1
#1 0x0e118da145f8 in ARCFOUR_string_to_key (context=0x44f9fba1a000,
enctype=KRB5_ENCTYPE_ARCFOUR_HMAC_MD5, password=..., salt=...,
opaque=..., key=0x44f9fba211d8)
at /space/system/usr_src/crypto/heimdal/lib/krb5/salt-arcfour.c:84
84 EVP_DigestUpdate (m, , 1);
(gdb) list
79
80 /* LE encoding */
81 for (i = 0; i < len; i++) {
82 unsigned char p;
83 p = (s[i] & 0xff);
84 EVP_DigestUpdate (m, , 1);
85 p = (s[i] >> 8) & 0xff;
86 EVP_DigestUpdate (m, , 1);
87 }
88
(gdb) print i
$1 = 0
(gdb) print len
$2 =
(gdb) print p
$3 = 49 '1'
(gdb) print m
$4 = (EVP_MD_CTX *) 0x43e31de4bc60
(gdb) print *m
$5 = {reqdigest = 0x17e678afd470, digest = 0x0, engine = 0x0, flags = 0,
md_data = 0x0, pctx = 0x0, update = 0x0, algctx = 0x0, fetched_digest =
0x0}
(gdb)
---snip---
Bye,
Alexander.
--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org : PGP 0x8F31830F9F2772BF
signature.asc
Description: OpenPGP digital signature
Re: crashinfo fails on 'version'
On Mon, 2 Oct 2023 20:15:30 + (UTC) "Bjoern A. Zeeb" wrote: > On Fri, 29 Sep 2023, Bjoern A. Zeeb wrote: > > > On Fri, 29 Sep 2023, Warner Losh wrote: > > > >> On Fri, Sep 29, 2023, 3:56 PM Bjoern A. Zeeb > >> > >> wrote: > >> > >>> Hi, > >>> > >>> saveinfo wrote a good core, but crashinfo bails on it: > >>> > >>> 'version' has unknown type; cast it to its declared type > >>> 'version' has unknown type; cast it to its declared type > >>> Unable to find matching kernel for /var/crash/vmcore.1 > >>> > >>> I've got gdb-13.1_3 installed. > >>> > >>> Anyone any insights? > >>> > >> > >> What does kgdb say? Iirc crashinfo uses lldb for this. > > > > I would hope the latter was true but: > > > > % grep -i lldb /usr/sbin/crashinfo % > > > > at least on the version here still very much gdb. > > > > > > I have started to trace and manually called the same commands crashinfo > > seems to run (without batch). > > > > It seems the real problem might be that it cannot find debug symbols > > inside the VM. > > > > I am wondering if there was a way to check for that to make the error > > messages less cryptic (e.g., maintenance print symbols empty if it does > > what I would think; I'll hopefully see soon). > > > > Also turns out /etc/src.conf on the provided host had WITHOUT_DEBUG_FILES= > > and even building and installing with -DWITH_DEBUG_FILES does not seem > > to override that (probably by design), which likely was the reason for > > no symbols in first place. > > > > > > I'll hopefully have more answers in 20 minutes. Remote debugging wifi > > less fun than I hoped. > > Well things in src.conf commented out, also tried with > -DWITHOUT_SPLIT_KERNEL_DEBUG, ... I still get no debugging symbols > with a main amd64 GENERIC installed. Neither in /usr/lib/debug not > /boot/kernel/kernel. I am totally confused as to why currently. > > Build host is 14.0-ALPHA1 from mid August. > > Anyone any idea? > This is weird since GENERIC has: GENERIC:makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols Maybe you also need makeoptions DEBUG+=-O0 to get all the debug symbols? -- Gary Jennejohn
