Re: Defaulting serial communication to 115200 bps for FreeBSD 14
On Wed, Aug 16, 2023, 9:38 PM Dennis Clarke wrote: > On 8/16/23 22:28, Alexander Motin wrote: > > On 16.08.2023 18:14, Dennis Clarke wrote: > >> The default serial communications config on most telecom equipment that > >> I have seen ( in the last forty years ) defaults to 9600 8n1. If people > >> want something faster from FreeBSD then do the trivial : > >> > >> set comconsole_speed="115200" > >> set console="comconsole" > >> > >> Is that not trivial enough? > > > > Except it is not a telecom equipment 40 years ago. Even at 115200 that > > I routinely use on my development systems I feel serial console output > > affects verbose boot time and kernel console debugging output. I also > > have BIOS console redirection enabled on my systems, and I believe the > > default there is also 115200, and even that is pretty slow. I see no > > point to stay compatible if it is unusable. > > > > You seem to be missing the point. > > You need to make a configuration choice. You. Not the world. You. > > Edit your /boot/loader.conf and put in the lines above. > > Then be happy. > > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > GreyBeard and suspenders optional > > PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of > equipment. > Yes. Some tiny number of things has that as a default, an even larger number of things have a default of 115200 or even faster. And have had that default since the 90s. The whole point of defaults is that they reflect the needs of the most people. FreeBSD's defaults were already starting to be dated in 1.0... today almost everyone changes the defaults to the new value we are advocating. This is to make FreeBSD more useful out of the box to more people. To turn your argument around: people wanting the old defaults can configure their systems easily enough. If we look purely at the numbers, vastly fewer people withh be inconvenienced at 115200 than at 9600. People can still use 9600... that's likely never going away... this is just a more sensible default. Warner >
Re: Defaulting serial communication to 115200 bps for FreeBSD 14
On 8/16/23 22:28, Alexander Motin wrote: On 16.08.2023 18:14, Dennis Clarke wrote: The default serial communications config on most telecom equipment that I have seen ( in the last forty years ) defaults to 9600 8n1. If people want something faster from FreeBSD then do the trivial : set comconsole_speed="115200" set console="comconsole" Is that not trivial enough? Except it is not a telecom equipment 40 years ago. Even at 115200 that I routinely use on my development systems I feel serial console output affects verbose boot time and kernel console debugging output. I also have BIOS console redirection enabled on my systems, and I believe the default there is also 115200, and even that is pretty slow. I see no point to stay compatible if it is unusable. You seem to be missing the point. You need to make a configuration choice. You. Not the world. You. Edit your /boot/loader.conf and put in the lines above. Then be happy. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of equipment.
Re: Defaulting serial communication to 115200 bps for FreeBSD 14
On 16.08.2023 18:14, Dennis Clarke wrote: The default serial communications config on most telecom equipment that I have seen ( in the last forty years ) defaults to 9600 8n1. If people want something faster from FreeBSD then do the trivial : set comconsole_speed="115200" set console="comconsole" Is that not trivial enough? Except it is not a telecom equipment 40 years ago. Even at 115200 that I routinely use on my development systems I feel serial console output affects verbose boot time and kernel console debugging output. I also have BIOS console redirection enabled on my systems, and I believe the default there is also 115200, and even that is pretty slow. I see no point to stay compatible if it is unusable. -- Alexander Motin
Re: Defaulting serial communication to 115200 bps for FreeBSD 14
On 8/16/23 03:10, Tomoaki AOKI wrote: On Tue, 15 Aug 2023 21:02:57 -0700 Cy Schubert wrote: Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 message dated "Tue, 15 Aug 2023 17:18:37 -0400." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii In message , Ed Maste writes: FreeBSD currently uses 9600 bps as the default for serial communication -- in the boot loader, kernel serial console, /etc/ttys, and so on. This was consistent with most equipment in the 90s, when these defaults were established. Today 115200 bps seems to be much more common, and I'm proposing that we make it the default for FreeBSD 14.0. I have a review open: https://reviews.freebsd.org/D36295. There are a few minor nits in the review to be addressed still but assuming there's general agreement I'll iterate on those and commit this in a few logical chunks. There should probably be an UPDATING entry for those who use boot0 to revert back to 9600 in that case. Not read the diff though, if the baud rate is re-initialized in boot1 or later (including btx, loader, kernel and userland) and handshake again, most of the boot process and later would run at 115200 bps. The default serial communications config on most telecom equipment that I have seen ( in the last forty years ) defaults to 9600 8n1. If people want something faster from FreeBSD then do the trivial : set comconsole_speed="115200" set console="comconsole" Is that not trivial enough? Also, merely a funny observation, if one tries a baud rate lower than 9600 then FreeBSD will panic. Jut a funny thing I have seen over and over. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional
HEADS UP: $FreeBSD$ Removed from main
Greetings, I've just pushed the results that remove 31,035 instances of $FreeBSD$ from the main branch. >From this point out, we are effectively done with $FreeBSD$, though there's 4 places where $FreeBSD$ still exists in the tree: (1) In the contrib area, we have 621 remaining. I didn't remove any from contrib code. (2) There's 140 $FreeBSD$ or similar tags part of 'static const char rcsid[] = "$FreeBSD$". There's too many different styles to get with automation, and they are also intertwingled with sccsid[] entries, other SCCS tags and copyright strings. These will be sorted out separately, since we need to talk about old sccs tags in the tree and sort that out too at the same time. (3) There's one $FreeBSD$ in a version tag for bootinfo for chrp bootinfo.txt. I don't know the effects of removing this entry, so I left it in place. Not sure it's worth fixing. (4) indent tests remove $FreeBSD$ so that it doesn't screw them up. This is likely harmless, but could be removed. I didn't want to mess with it, though. I've removed 99.5%+ of the 'live' instances in the tree. The ones from contrib should be removed upstream and I plan to do a MFC-like thing where I cherry-pick some commits, and run my script against stable/13 and include the main hash so our scripting things it's a real MFC (the diff hashes will differ, so git's native tooling will take the slow path, at least, and may get confused otherwise. I've built world on aarch64 and amd64. I've built kernels on all the architectures after this change... or well, last night's main. I eye-balled today's changes and they all look good, but there's no incremental building with this change, so I'm starting a new universe after I loop the change back into my tree. Also: expect long build times, git fetch times, etc after this. Thanks to the many people who gave me feedback on the details of this change and how to chunk it up. Hopefully the 40ish commits was the right balance between 'all at once' and 'every dir'. As always, if there's any problems after this change, please let me know. Warner
Re: Speed improvements in ZFS
Am 2023-08-15 23:29, schrieb Mateusz Guzik: On 8/15/23, Alexander Leidinger wrote: Am 2023-08-15 14:41, schrieb Mateusz Guzik: With this in mind can you provide: sysctl kern.maxvnodes vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes vfs.recycles_free vfs.recycles After a reboot: kern.maxvnodes: 10485760 vfs.wantfreevnodes: 2621440 vfs.freevnodes: 24696 vfs.vnodes_created: 1658162 vfs.numvnodes: 173937 vfs.recycles_free: 0 vfs.recycles: 0 New values after one rund of periodic: kern.maxvnodes: 10485760 vfs.wantfreevnodes: 2621440 vfs.freevnodes: 356202 vfs.vnodes_created: 427696288 vfs.numvnodes: 532620 vfs.recycles_free: 20213257 vfs.recycles: 0 Meanwhile if there is tons of recycles, you can damage control by bumping kern.maxvnodes. What's the difference between recycles and recycles_free? Does the above count as bumping the maxvnodes? Looks like there are not much free directly after the reboot. I will check the values tomorrow after the periodic run again and maybe increase by 10 or 100 so see if it makes a difference. If this is not the problem you can use dtrace to figure it out. dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or something else? I mean checking where find is spending time instead of speculating. There is no productized way to do it so to speak, but the following crapper should be good enough: [script] I will let it run this night. Bye, Alexander. -- http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.orgnetch...@freebsd.org : PGP 0x8F31830F9F2772BF
Re: www/chromium will not build on a host w/ 8 CPU and 16G mem
Thanks for all the hints I got so far. I started already the build with MAKE_JOBS_UNSAFE=yes. It's running now for some 8 hours and has built 32% (as the log says). So it will need some 16 hours more... -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub
Re: www/chromium will not build on a host w/ 8 CPU and 16G mem
On Tue, 15 Aug 2023 23:19:37 -0700 Mark Millard wrote: > Matthias Apitz wrote on > Date: Wed, 16 Aug 2023 04:31:38 UTC : > > > I have built ~2200 ports successful on my build server, which is a > > Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory: > > > > Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz > > (3292.74-MHz K8-class CPU) > > Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 > > CPUs > > Aug 11 19:03:21 jet kernel: avail memory = 16582250496 (15814 MB) > > > > I have set swap to 4GB + 10GB + 10GB: > > > > # swapctl -lh > > Device: Bytes Used: > > /dev/da0p3 4.0G 1.5G > > /dev/md9 10G 1.5G > > /dev/md10 10G 1.5G > > Are those /dev/md* vnode backed? If not, what type are they? > > FYI: On 2017-Feb-13, at 7:20 PM, Konstantin Belousov > wrote on the freebsd-arm list: > > QUOTE > . . . > > swapfile write requires the write request to come through the filesystem > write path, which might require the filesystem to allocate more memory > and read some data. E.g. it is known that any ZFS write request > allocates memory, and that write request on large UFS file might require > allocating and reading an indirect block buffer to find the block number > of the written block, if the indirect block was not yet read. > > As result, swapfile swapping is more prone to the trivial and unavoidable > deadlocks where the pagedaemon thread, which produces free memory, needs > more free memory to make a progress. Swap write on the raw partition over > simple partitioning scheme directly over HBA are usually safe, while e.g. > zfs over geli over umass is the worst construction. > END QUOTE > > Use of swap partitions is to be recommended to minimize the chance of > paging related deadlocks. > > You have not reported on how much swap was in use shortly before the > "was killed: a thread waited too long to allocate a page" notice. > (After the notice is too late of a time frame to be of interest.) > > > and poudriere does use ZFS. > > > > Building the last outstanding port www/chromium in single job mode > > fails reproducible with a kernel message: > > > > Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was killed: > > a thread waited too long to allocate a page > > You have not been explicit about how you have managed > RAM+SWAP tradeoffs in /usr/local/etc/poudriere.conf > settings. > > In /usr/local/etc/poudriere.conf : what are you using > as the USE_TMPFS value: > > # Use tmpfs(5) > # This can be a space-separated list of options: > # wrkdir- Use tmpfs(5) for port building WRKDIRPREFIX > # data - Use tmpfs(5) for poudriere cache/temp build data > # localbase - Use tmpfs(5) for LOCALBASE (installing ports for > packaging/testing) > # all - Run the entire build in memory, including builder jails. > # yes - Enables tmpfs(5) for wrkdir and data > # no- Disable use of tmpfs(5) > > Only 2 of the options keep the tmpfs use small: > > data > no > > For example, building rust can use 10 GiBytes+ of just tmpfs > space that competes for RAM+SWAP. > > The last personal I helped that was getting "a thread waited > too long to allocate a page" it turned out that changing the > USE_TMPFS= assignment to one of the 2 options eliminated the > issue. (No guarnatee of such here.) [There were 2 USE_TMPFS= > assignments, the 2nd "winning" --but the first had been > intended.] > > Are you defining ALLOW_MAKE_JOBS= ? If yes, are you using > /usr/local/etc/poudriere.d/make.conf (or the like) to assign > MAKE_JOBS_NUMBER= (or the like)? If yes, to what? Last I knew > the official port -> package builders use MAKE_JOBS_NUMBER=2 > for their activity with ALLOW_MAKE_JOBS in use. > > A similar point goes for if you are assigning > ALLOW_MAKE_JOBS_PACKAGES= . Are you? > > These can contribute to RAM+SWAP usage and higher load > averages. > > > What could I do in the port's options or kernel values? > > I've not actually gone in either of those directions above. > But nothing at this point says if I've happened to cover > your case vs. not. > > === > Mark Millard > marklmi at yahoo.com Just a FYI: www/chromium built fine (stable/13, though) with poudriere. RAM is 32GB and 64GB single dedicated swap partition on SSD. Using ports-mgmt/poudriere-devel. No ccache used. In /usr/local/etc/poudriere.conf, USE_TMPFS=yes ALLOW_MAKE_JOBS=yes The line below is kept commented out. #TMPFS_LIMIT=8 On main (booted from different physical drive on the same PC), I don't use poudriere[-devel], but www/chromium (previous version) built fine using ports-mgmt/pkg_replace. The size of /tmp (swap-backed tmpfs) is not limited (means at maximum of 64GB). -- Tomoaki AOKI
Re: Defaulting serial communication to 115200 bps for FreeBSD 14
On Tue, 15 Aug 2023 21:02:57 -0700 Cy Schubert wrote: > Cheers, > Cy Schubert > FreeBSD UNIX: Web: https://FreeBSD.org > NTP: Web: https://nwtime.org > > e^(i*pi)+1=0 > >message dated "Tue, 15 Aug 2023 17:18:37 -0400." > Mime-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > > In message om> > , Ed Maste writes: > > FreeBSD currently uses 9600 bps as the default for serial > > communication -- in the boot loader, kernel serial console, /etc/ttys, > > and so on. This was consistent with most equipment in the 90s, when > > these defaults were established. Today 115200 bps seems to be much > > more common, and I'm proposing that we make it the default for FreeBSD > > 14.0. > > > > I have a review open: https://reviews.freebsd.org/D36295. There are a > > few minor nits in the review to be addressed still but assuming > > there's general agreement I'll iterate on those and commit this in a > > few logical chunks. > > > > There should probably be an UPDATING entry for those who use boot0 to > revert back to 9600 in that case. Not read the diff though, if the baud rate is re-initialized in boot1 or later (including btx, loader, kernel and userland) and handshake again, most of the boot process and later would run at 115200 bps. -- Tomoaki AOKI
Re: www/chromium will not build on a host w/ 8 CPU and 16G mem
Matthias Apitz wrote on Date: Wed, 16 Aug 2023 04:31:38 UTC : > I have built ~2200 ports successful on my build server, which is a > Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory: > > Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz > (3292.74-MHz K8-class CPU) > Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 > CPUs > Aug 11 19:03:21 jet kernel: avail memory = 16582250496 (15814 MB) > > I have set swap to 4GB + 10GB + 10GB: > > # swapctl -lh > Device: Bytes Used: > /dev/da0p3 4.0G 1.5G > /dev/md9 10G 1.5G > /dev/md10 10G 1.5G Are those /dev/md* vnode backed? If not, what type are they? FYI: On 2017-Feb-13, at 7:20 PM, Konstantin Belousov wrote on the freebsd-arm list: QUOTE . . . swapfile write requires the write request to come through the filesystem write path, which might require the filesystem to allocate more memory and read some data. E.g. it is known that any ZFS write request allocates memory, and that write request on large UFS file might require allocating and reading an indirect block buffer to find the block number of the written block, if the indirect block was not yet read. As result, swapfile swapping is more prone to the trivial and unavoidable deadlocks where the pagedaemon thread, which produces free memory, needs more free memory to make a progress. Swap write on the raw partition over simple partitioning scheme directly over HBA are usually safe, while e.g. zfs over geli over umass is the worst construction. END QUOTE Use of swap partitions is to be recommended to minimize the chance of paging related deadlocks. You have not reported on how much swap was in use shortly before the "was killed: a thread waited too long to allocate a page" notice. (After the notice is too late of a time frame to be of interest.) > and poudriere does use ZFS. > > Building the last outstanding port www/chromium in single job mode > fails reproducible with a kernel message: > > Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was killed: > a thread waited too long to allocate a page You have not been explicit about how you have managed RAM+SWAP tradeoffs in /usr/local/etc/poudriere.conf settings. In /usr/local/etc/poudriere.conf : what are you using as the USE_TMPFS value: # Use tmpfs(5) # This can be a space-separated list of options: # wrkdir- Use tmpfs(5) for port building WRKDIRPREFIX # data - Use tmpfs(5) for poudriere cache/temp build data # localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing) # all - Run the entire build in memory, including builder jails. # yes - Enables tmpfs(5) for wrkdir and data # no- Disable use of tmpfs(5) Only 2 of the options keep the tmpfs use small: data no For example, building rust can use 10 GiBytes+ of just tmpfs space that competes for RAM+SWAP. The last personal I helped that was getting "a thread waited too long to allocate a page" it turned out that changing the USE_TMPFS= assignment to one of the 2 options eliminated the issue. (No guarnatee of such here.) [There were 2 USE_TMPFS= assignments, the 2nd "winning" --but the first had been intended.] Are you defining ALLOW_MAKE_JOBS= ? If yes, are you using /usr/local/etc/poudriere.d/make.conf (or the like) to assign MAKE_JOBS_NUMBER= (or the like)? If yes, to what? Last I knew the official port -> package builders use MAKE_JOBS_NUMBER=2 for their activity with ALLOW_MAKE_JOBS in use. A similar point goes for if you are assigning ALLOW_MAKE_JOBS_PACKAGES= . Are you? These can contribute to RAM+SWAP usage and higher load averages. > What could I do in the port's options or kernel values? I've not actually gone in either of those directions above. But nothing at this point says if I've happened to cover your case vs. not. === Mark Millard marklmi at yahoo.com
