Re: Using mmap(2) in sort(1) instead of temp files
On Thu, Apr 04, 2024 at 12:02:30PM +, Ice Cream wrote: > Given the issues about using mmap, can anybody suggest how > I should proceed with the implementation, or if I should at all? There are two potential ways where mmap(2) could help improve the speed of sort: - If you know the input file name, use a read-only mmap() of that file and avoid all buffering. Downside: you can not store \0 at the end of a line anymore and need to deal with char*/size_t pairs for strings. - You use "swap space" instead of a temporary file by doing mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_ANNON, -1, 0); and then use the returned pointer for temporary stuff. Obviously you can not do that for arbitarary sizes, so maybe you have to keep the old code and only do this trick if the file size is small enough or you process the file in pieces or whatever. Since the original comment hints at "instead of temp files" it is pretty clear that the second variant is meant. This avoids all file system operations and if the machine you run on has enough free memory it might not even actually touch swap space. Martin
Re: sysinst username length limit
On Wed, Apr 03, 2024 at 10:53:58AM +0200, Hauke Fath wrote:
> > The limit still exists if you use NIS to manage passwords.
>
> It does? My YP master says
>
> % awk -F: '{ if (length($1) > 8) { print $1 }}' < /etc/passwd | wc -l
> 18
> %
>
> The linux and freebsd clients seem to be fine with that.
>
> I don't use {passwd,group}_compat, though.
I don't know for sure, I don't use it myself.
IIRC you could always use longer names if they were unique within the first
8 chars.
Maybe it would be ok to ignore any such limits anyway since in setups where
they apply you would either know about it (you are the admin) or would get
your login assigned by the admin, so caring about it at the installer level
is not very helpful.
Martin
Re: sysinst username length limit
On Wed, Apr 03, 2024 at 05:34:21AM +, Charlotte Koch wrote: > The 8 character limit is probably some historical thing, and besides, > it's completely bogus. I create accounts called 'charlotte' (9 chars) > all the time, with and without sysinst. The limit still exists if you use NIS to manage passwords. Martin
Re: numpy failure on netbsd-9, rintl
On Thu, Nov 16, 2023 at 09:03:22AM -0500, Greg Troxel wrote: > +#ifndef __HAVE_LONG_DOUBLE > +__strong_alias(_frexpl, frexp) > +__weak_alias(frexpl, _frexpl) > +#endif That is exactly the right thing to do (and we have added such aliases to several functions already, but not done a systematic sweep). If double == long double we need an alias to make the long double version of the function resolve to the plain double one. Many NetBSD architectures are affected by this, but the libm build system is ... confusing. Would be a good idea to create an ATF test program that refers to all of them, so we can easily catch all missing ones at build time (like we did with several atomics/sync primitives some time ago). Martin
Re: new certificate stuff
On Sun, Aug 27, 2023 at 10:53:58PM +, Taylor R Campbell wrote: > Currently, if /etc/openssl/certs.conf doesn't exist, `certctl rehash' > (the crux of `postinstall fix opensslcerts') will print an error > message and then exit with status 0. This combination is a bug -- > need to think a bit about it, but probably better to exit nonzero than > to suppress the error message. Sounds good to me. Make the error message point at some docs and ask the user to - deinstall mozilla-rootcerts - manually copy /etc/openssl/certs.conf over or - add a manual certs.conf and keep using their old certificate setup. and then re-run postinstall. > So if you unpack new _non-etc_ sets, `postinstall fix' won't > clobber your /etc/openssl/certs directory. This is good. Martin
Re: Android-like NetBSD
On Mon, Aug 21, 2023 at 05:57:31AM -0400, Greg Troxel wrote: > But interesting point about not having a proper destdir, and the > presence of stub libraries. That part is pretty simple: just download the comp.tgz set for the target and extract only a few paths from it, like: ./lib/ ./usr/lib/ ./include/ But then you probably need to do a "build.sh tools" for it (which requires the full source tree). We could create a "tools.tgz" for it, but that would be host specific and likely not worth the bandwidth/space. Martin
Re: Architecture neutral packages (mozilla-rootcerts-openssl)
On Thu, Aug 03, 2023 at 02:31:39PM +, John Klos wrote: > How it ultimately happens is up to people who understand things better than > I do, but what whould be lovely to see would be: > > 1) a way to install rootcerts in sysinst > > 2) a way to install them post-install, and/or update them > > 3) an easy way for people who have reasons to be deliberate to allow / >block certain certs so that updates don't undo their work That is exactly what I'd like to see too. I will do (1) immediately once the other two are designed/agreed. I also think that this is NOT a pkgsrc problem, BUT pkgsrc has two very nice property when it comes to (2): the quaterly release cycle, and the independence from the NetBSD release branches. The trust anchors are not really related to any NetBSD branches, so having a global set in -current and always making sure that any changes to it get propagated immediately to all branches is doable but seems a bit like wasted work. And a typical thing that users do is: grab the latest official release (currently that would be the quite ancient 9.3). If the installer offers and on request installs the historic set of trust anchors from that release time, the user might miss quite a few changes. If instead it used something pkgsrc-ish they would get a maximal 3 month outdated version. So using something from pkgsrc, somehow treated specially, and put in a constant place (or: published with a constant URL) in every quaterly release would be nice for installations with network access and for (2) later (automatic) updates. The special handling does not have to happen inside pkgsrc at all - we could have a script that unpacks the amd64 mozilla root certs binary pkg once a new version has been uploaded (or the quaterly symlink switched) and put that data in an arbitrary format (that is understood by sysinst and the [not yet existing] auto-update script) at the fixed URL. In summary I'd be happy to rely on pkgsrc for this, but I don't think we need pkgsrc to make any changes. Martin
Re: sed(1) and LC_CTYPE
On Wed, Jul 26, 2023 at 12:19:39PM -0400, Mouse wrote: > > $ export LC_CTYPE=fr_FR.ISO8859-15 > > > $ echo "éé" | sed 's/é/\/g' > > sed: 1: "s/é/\/g": RE error: trailing backslash (\) > > I agree that's broken. > > > Since, to my knowledge, we do not support anything via iconv or > > whatever, shouldn't we assume simply a string of bytes \`a la C, that > > is: > > Seems to me there's a deeper problem. Even if something like iconv > _were_ available, fr_FR.ISO8859-15 is a single-octet character set, so > > > - (void) setlocale(LC_ALL, ""); > > + (void) setlocale(LC_ALL, "POSIX"); > > should, it seems to me, make no difference. Am I misunderstanding? Indeed - and it only does on architectures where char == signed char: $ export LC_CTYPE=fr_FR.ISO8859-15 $ echo "éé" | sed 's/é/\/g' sed: 1: "s/é/\/g": RE error: trailing backslash (\) $ uname -a NetBSD seven-days-to-the-wolves.aprisoft.de 10.99.6 NetBSD 10.99.6 (GENERIC) #642: Wed Jul 26 10:32:32 CEST 2023 mar...@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/amd64/compile/GENERIC amd64 but: $ export LC_CTYPE=fr_FR.ISO8859-15 $ echo "éé" | sed 's/é/\/g' $ uname -a NetBSD big-apple.aprisoft.de 10.99.4 NetBSD 10.99.4 (POWERMAC_G5.MP) #100: Tue Jun 27 19:48:49 CEST 2023 mar...@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/macppc/compile/POWERMAC_G5.MP macppc Fun :-) Thierry, can you please file a PR? Martin
Re: DRM/KMS
On Fri, Jul 07, 2023 at 03:08:25PM +0100, David Brownlee wrote: > So far some good changes (cribbing shamelessly from other suggestions) might > be: > - Implement "boot -D" (or similar) to boot with all DRM disabled, to > make it easier for hardware with issues That should be quite easy. > - Allow optionally initialising DRM after boot and transferring > console ownership (may add more work in future upgrades, but works > well with above item :) > - Rework wsdisplay to try to reduce abstraction violations and make it > cleaner to work with I think all hands should help debugging what we have now, and help with the pending next update ASAP. > - Looking at issues with certain hardware (can probably find systems > to ship if anyone is interested...) Well, often it is not exactly clear *what* makes the affected system specific. Martin
Re: DRM/KMS
On Fri, Jul 07, 2023 at 02:30:14PM +0100, David Brownlee wrote: > drm/kms definitely is hugely complicated, overly Linux focussed, and > difficult to maintain and update. A lot of effort has been put into > getting it to run on NetBSD (and updating from previous versions), but > it's currently the only viable game in town. I also thing it is not *that* far away from working fine. The releng wiki page lists a bunch of PRs against it, but those are mostly hard to fix because the problem only happens on *some* hardware, and sometimes only in special scenarios (e.g. serial console used and the monitor powered on during drm/kms attaching). That it all is a mess we probably all agree with. And this will require more updates, every year - GPU hardware does evolve, and available options change. Using no drm/kms is a good alternative (and works great on NetBSD in general). But you loose WebGL and sometimes accalerated video playback, and also often support for mulitple displays (but that part might even be easy to fix). Martin
Re: Trivial program size inflation
On Sun, Jul 02, 2023 at 10:53:31PM +, Emmanuel Dreyfus wrote: > On Sun, Jul 02, 2023 at 11:16:12PM +0200, Joerg Sonnenberger wrote: > > > At least in 9.3, dlopen() in a static binary does not work. Try using > > > a NSS module from a statically lunked binary to check that. > > > > It does work in the sense that it always fails. > > It fails silently, there not even an unhelpful error message. dlerror() returns "Service unavailable" Martin
Re: Trivial program size inflation
On Sun, Jul 02, 2023 at 01:49:57PM -0400, Mouse wrote: > Why? If I'm looking at overhead size, I am most interested in just the > overhead size, which is exactly what a no-op program gives. If I want > to look at the overhead of printf, or malloc, or something, I'd use a > program that just calls those. That might be interesting, but it's not > what I was doing here. I would define overhead as "anything that is not needed for what this concrete program does" and that is always ~100% in your test, but gets blurry quickly for all real programs. With most real world programs (hopefully) nearly 100% of what you see as overhead now is actually needed - and it still may be bigger than what we hope for due to suboptimal modularization. I am all for reducing sizes for real world things (linked either dynamic or static) wherever possible, and I'm pretty sure NetBSD will gladly accept patches to make them smaller if they are technically sound and don't impact standards compliance. Martin
Re: Trivial program size inflation
On Sun, Jul 02, 2023 at 12:34:25PM +0200, tlaro...@polynum.com wrote: > It is curious that you react this way in a thread where, you as others, > have had your jaw drop seeing the size of a literally do_nothing > executable. This was unseen precisely because few use static linking. I'm not sure that is what we see here. As Jason said: we have one platform (sun2) where only static linking is available, and we do notice huge size explosions, e.g. when install media does not fit any more. This platform is part of the daily builds. Note that a "do nothing" binary is a useless tech demo. If you really want it, you could avoid libc and csu, and be down to a few bytes. The "jemalloc is big" details may be the only suprising thing here, but some did know that and we provide a special build option to use an older (and much smaller) version instead. Way more interesting than useless tech demo sizes would size inflation of a real world minimal program, when linked statically. The other things that we *might* look into (if someone volunteers) is to better modularize the CSU code, but it is not immediately clear how that could/should be done. However, I personally disagree with Jason on the static linking support and would prefer to keep it. I find it usefull every now and then in special situations, and I really like that I can test boot NetBSD kernels w/o swapping libc to a matching version. It does not come for free though. Martin
Re: printf(1), sh(1), POSIX.2 and octal escape sequences
On Wed, Jun 28, 2023 at 05:59:10PM +0200, tlaro...@polynum.com wrote: > "\ddd", where ddd is a one, two, or three-digit octal number, shall be > written as a byte with the numeric value specified by the octal number." > > ? Because I parse it as: an octal escape sequence can be \d, or \dd or > \ddd; and the result is a byte value. Exactly. But for the parser the "byte value" is irrelevant, that part is semantics (and checked later). Syntactically you write an octal number with upto three digits. If you want to write a two digit octal number you can not continue with another ocatal digit. In C you could do "...\77" "7" and have it concat the literals. In config files (without concatenation) you need some other trick. Martin
Re: printf(1), sh(1), POSIX.2 and octal escape sequences
On Wed, Jun 28, 2023 at 05:01:46PM +0200, tlaro...@polynum.com wrote: > But you can't: from the syntax given, \777 is a perfectly valid \77 > octal sequence followed by the character '7'. No, from the Posix text you quoted it clearly is a three digit ocatl sequence, and its value is out of range. Martin
Re: vi(1) one line crasher
On Thu, Jun 22, 2023 at 08:52:12PM +0200, Martin Husemann wrote:
> env EXINIT='set nu showmatch ts=8 wl=72' vi /tmp/vi_crasher.txt
>
> does crash for me when typing $
#2 0x009a54da in vs_paint (sp=sp@entry=0x6fb6f40de000,
flags=flags@entry=3) at /work/src/external/bsd/nvi/dist/vi/vs_refresh.c:726
726 abort(); /* XXX infinite recursion */
(gdb) list
721 abort();
722 }
723 #else
724 if (vip->sc_smap == NULL) {
725 if (F_ISSET(sp, SC_SCR_REFORMAT))
726 abort(); /* XXX infinite recursion */
727 F_SET(sp, SC_SCR_REFORMAT);
728 return (vs_paint(sp, flags));
729 }
730 #endif
#3 0x009983de in vs_paint (sp=sp@entry=0x6fb6f40de000, flags=3)
at /work/src/external/bsd/nvi/dist/vi/vs_refresh.c:728
#4 0x009990d5 in vs_refresh (sp=sp@entry=0x6fb6f40de000,
forcepaint=forcepaint@entry=0)
at /work/src/external/bsd/nvi/dist/vi/vs_refresh.c:99
#5 0x009941e6 in vi (spp=spp@entry=0x7f7fffe31740)
at /work/src/external/bsd/nvi/dist/vi/vi.c:115
#6 0x0097bdf8 in editor (wp=wp@entry=0x6fb6f40f1000,
argc=, argc@entry=2, argv=,
argv@entry=0x7f7fffe319c8)
at /work/src/external/bsd/nvi/dist/common/main.c:436
#7 0x009a5864 in main (argc=2, argv=0x7f7fffe319c8)
at /work/src/external/bsd/nvi/dist/cl/cl_main.c:134
Can you file a PR please?
Thanks!
Martin
Re: vi(1) one line crasher
On Thu, Jun 22, 2023 at 08:46:44PM +0200, tlaro...@polynum.com wrote: > So wl=72 and ts=8 may reproduce for you? Yes! env EXINIT='set nu showmatch ts=8 wl=72' vi /tmp/vi_crasher.txt does crash for me when typing $ Martin
Re: vi(1) one line crasher
On Thu, Jun 22, 2023 at 06:22:42PM +0200, tlaro...@polynum.com wrote: > If one opens this one line script in vi, in an xterm with default size, > using the dollar '$' to go to the end of the line crashes vi(1). I can't reproduce this. Which version of NetBSD? The in-tree vi? What is your $TERM? Which xterm? Martin
Re: style, sysexits(3), and man RETURN VALUES for sys programs
On Sat, Jun 03, 2023 at 01:45:44PM +0200, tlaro...@polynum.com wrote: > Furthermore, I'm adding a RETURN VALUES section to inetd.8 and I think > it should be standard practice for sys programs. That is for functions returning a value, the proper .Sh here would be EXIT STATUS. Martin
Re: NetBSD 10 and NetBSD 11...
On Wed, May 17, 2023 at 08:47:33PM +0200, tlaro...@polynum.com wrote: > But the fact that the advertised list of changes for 10 stops at > February 2023 is probably something the webmaster(s) should look > at: it's a bit confusing/disturbing... Where do you see that? Martin
Re: NetBSD 10 and NetBSD 11...
On Wed, May 17, 2023 at 11:29:43AM -0700, Jason Thorpe wrote: > There have been a steady stream of bug fixes from the trunk being > pulled into the netbsd-10 release branch. I don't have knowledge of > releng@'s plans vis a vis a release date. You can find details about the 10.0 release state at https://wiki.netbsd.org/releng/netbsd-10/ (but right now that site is not reachable for me) Martin
Re: using gas/gld from devel/binutils
On Wed, May 17, 2023 at 11:17:28AM +0200, Edgar Fuß wrote: > What Version of binutils do -9, -10 and -current use? Are there patches in > base that are missing in pkgsrc? 2.31, 2.34 and 2.39 respectively. And probably yes (but I wouldn't expect anything relevant for amd64). Martin
Re: split(1): add '-c' to continue creating files
On Sun, Feb 12, 2023 at 04:05:20PM -0500, Jan Schaumann wrote: > The attached diff adds a flag "-c" (mnemonic "create, > don't overwrite" or "continue where you left off"): > > $ split file; ls > xaa xab xac xad > $ split -c second-file; ls > xaa xab xac xad xae xaf xag xah xai xaj I think this is a dangerous and non-obvious user interface, especially when we hit collisions later or data changes and we are re-doing the split. How about instead adding an option that sets the first name explicitly and keeps the "abort on failure" behaviour? Besides, isn't your intended behaviour easily done with: $ cat file second-file | split ? Martin
Re: tcsh as csh default
On Fri, Oct 21, 2022 at 03:06:41PM +0200, Havard Eidnes wrote: > There's a definite size difference between csh and tcsh, > approximately 2x -- this is on NetBSD/amd64 9.x, and I include > /bin/sh for comparison: > > $ ls -lf /usr/pkg/bin/tcsh /bin/csh /bin/sh > -rwxr-xr-x 1 root wheel 408688 Jun 28 03:57 /usr/pkg/bin/tcsh > -r-xr-xr-x 1 root wheel 186712 Aug 4 17:30 /bin/csh > -r-xr-xr-x 1 root wheel 223632 Aug 4 17:30 /bin/sh > $ size /usr/pkg/bin/tcsh /bin/csh /bin/sh >textdata bss dec hex filename > 382121 21158 86984 490263 77b17 /usr/pkg/bin/tcsh > 1448647113 21664 173641 2a649 /bin/csh > 17788162879928 194096 2f630 /bin/sh > $ > > The size difference might not matter much for modern machines > (and, yes, these are all dynamically linked). Part of the difference is that both our /bin/sh and our /bin/csh are linked against libedit, while tcsh has similar code built in. Here are numbers from a sh4 machine running -current (where size does matter): [~] martin@last-of-the-heroes > ls -lf /usr/pkg/bin/tcsh /bin/csh /bin/sh -rwxr-xr-x 1 root wheel 317548 Jul 21 21:40 /usr/pkg/bin/tcsh* -r-xr-xr-x 1 root wheel 172408 Oct 18 08:14 /bin/csh* -r-xr-xr-x 1 root wheel 200032 Oct 18 08:15 /bin/sh* [~] martin@last-of-the-heroes > size /usr/pkg/bin/tcsh /bin/csh /bin/sh textdata bss dec hex filename 303581 11942 81684 397207 60f97 /usr/pkg/bin/tcsh 1427935237 20356 168386 291c2 /bin/csh 17116136205764 180545 2c141 /bin/sh [~] martin@last-of-the-heroes > ldd /bin/csh /bin/csh: -ledit.3 => /lib/libedit.so.3 -lterminfo.2 => /lib/libterminfo.so.2 -lc.12 => /lib/libc.so.12 -lgcc_s.1 => /lib/libgcc_s.so.1 -lutil.7 => /lib/libutil.so.7 [~] martin@last-of-the-heroes > size /lib/libedit.so.3.1 textdata bss dec hex filename 16012544527584 172161 2a081 /lib/libedit.so.3.1 Martin
Re: tcsh as csh default
On Fri, Oct 21, 2022 at 07:51:14AM +, Emmanuel Dreyfus wrote: > On Fri, Oct 21, 2022 at 07:39:45AM +0200, Martin Husemann wrote: > [csh] > > Or just remove it... > > It would be a bad joke for anyone that has set it as root shell. Oh, I did not mean to suggest marking it as "obsolete" and auto-removing the binary on updates, and certainly we would need some good update plan (docs, hints at upgrade time, ...) Martin
Re: tcsh as csh default
On Thu, Oct 20, 2022 at 11:57:52PM +, Bruno Melo wrote: > Any good reason to not update /bin/csh to tcsh as default csh like > FreeBSD and Dragonfly did? I think no one expect csh scripts > performance and use csh specially for interactive usage, so I can't see > a reason for not replacing to a csh enhanced and maintained version. Or just remove it... Martin
Re: grep -r: add default "."
On Sat, Sep 03, 2022 at 11:26:18AM +0200, Roland Illig wrote: > What about importing the current GNU grep, instead of patching the 2004 > version of it? That has a bad license. How about fixing the bsd version we have in tree instead? Maybe FreeBSD or OpenBSD already did that and we can borrwo? Martin
Re: [Christos Zoulas] CVS commit: src/usr.bin/ftp
On Fri, Sep 02, 2022 at 06:23:48PM +0300, Christos Zoulas wrote: > I think we should be installing the anchors by default. I also think > that people think that https gets validated by default. I agree. The problem is that we need to suply anchors now with new installations and have a way to keep them updated (and optionaly disabled). Could be something easy like using the mozilla root certs from last quarters pkgsrc branch, downloadable from some well known NetBSD.org URL. Either the original bin pkg and a (special) base system script to unpack and update (w/o pkg_add and architecture check), or some automatic magic on the server side to rebundle them in the format expected by the update script on the client side. And some local overrides (as we usually have with this kind of tools) to get a different bundle from a local URL or disable the whole update. Martin
Re: sh(1) and ksh(1) default PATH
On Sun, Aug 14, 2022 at 07:36:17AM +, nia wrote: > Display managers do not create login shells, and xdm was used. Adapt the default /etc/X11/xdm/Xsession instead? Maybe make it extract PATH from /etc/sh.rc ? Martin
Re: /rescue/tar needing liblzma.so.2
On Mon, Jul 04, 2022 at 06:58:39PM +, Emmanuel Dreyfus wrote: > On Mon, Jul 04, 2022 at 08:32:02PM +0200, Martin Husemann wrote: > > Try with PATH=/rescue, that should avoid the issue. > > Shouldn't /rescue binaries have /rescue prepended to their PATH? Tricky - isn't that too much magic? We could have the /rescue/sh default to /rescue/.profile or something and arrange good "rescue defaults" there. Martin
Re: /rescue/tar needing liblzma.so.2
On Mon, Jul 04, 2022 at 07:43:41PM +0200, Edgar Fuß wrote: > So why does something in /rescue need a shared library? It doesn't, but it exec's gzip via $PATH and finds /usr/bin/gzip - and that needs /usr/lib/liblzma.so.2. Try with PATH=/rescue, that should avoid the issue. Martin
Re: cmake core dumps in -6 emulation
On Fri, Jul 01, 2022 at 02:20:02PM +0200, Edgar Fuß wrote:
> > Ktrace it
> As mentioned, that doesn't work (well, it works, which is the problem).
>
> > there are 3 err() calls in pthread__init()
>
> Starting with
> #8 0x71b551460ac0 in err () from /usr/lib/libc.so.12
> #9 0x71b55240c47b in pthread__init () from /usr/lib/libpthread.so.1
> I disassembled a bit before 0x71b55240c47b, the relevant part being
> 0x71b55240c468 : lea0x56a(%rip),%rsi#
> 0x71b55240c9d9
> 0x71b55240c46f : mov$0x1,%edi
> 0x71b55240c474 : xor%eax,%eax
> 0x71b55240c476 : callq 0x71b552405eb0
> 0x71b55240c47b : mov%rdi,%rax
> and x/s 0x71b55240c9d9 says
> 0x71b55240c9d9: "mprotect stack"
>
> So what's going wrong?
Ah - probably inlining of the pthread__initmain() call, which does
base = (void *)(pthread__sp() & pthread__threadmask);
if ((pthread__sp() - (uintptr_t)base) < 4 * pagesize) {
pthread__mainbase = (vaddr_t)base;
base = STACK_GROW(base, pthread__stacksize);
pthread__mainstruct = (vaddr_t)base;
if (mprotect(base, pthread__stacksize,
PROT_READ|PROT_WRITE) == -1)
err(1, "mprotect stack");
}
size = pthread__stacksize;
and your ktrace shows:
23687 1 cmakeCALL mprotect(0x7f7fff80,0x40,3)
23687 1 cmakeRET mprotect -1 errno 13 Permission denied
and this sounds like PaX mprotect at work - you could test with paxctl
on the cmake file (or see below). It seems we did not enable
PAX_MPROTECT by default on amd64 before netbsd-8, so the old userland
might not be fully ready for it.
Maybe you should set sysctl security.pax.mprotect.global=0 while building
the old pkgs?
Martin
Re: cmake core dumps in -6 emulation
On Thu, Jun 30, 2022 at 07:29:18PM +0200, Edgar Fuß wrote: > #8 0x77a601860ac0 in err () from /usr/lib/libc.so.12 > #9 0x77a60280c47b in pthread__init () from /usr/lib/libpthread.so.1 Ktrace it - there are 3 err() calls in pthread__init(), one after sysctl hw.ncpu failed, one if _lwp_unpark_all fails and one after _lwp_ctl failed. Martin
Re: sysupgrade in src?
On Fri, Apr 15, 2022 at 08:21:00AM +, nia wrote: > There's been a few discussions over the years about the need for a > usable-over-ssh binary upgrade tool for NetBSD, but they haven't gone > very far. I don't object to your idea (and I have never used it). My own prefered method requires none of what you mentioned though (and I'm open for suggestions to improve it): (1) manually update the kernel (and modules if needed), reboot (2) run the already installed sysinst and select "Upgrade" and then "currently running system" (3) manually run etcupdate I have been considering to add a sysinst option to do (1), but it only is as straight forward as it sounds if you have no special boot setup. I would be interested to see how sysupgrade deals with that (and my guess is it won't work for most of my own machines). Properly doing (3) from within sysinst would be doable, but I personally find me tricking around and restarting over and over untill it is all good, and I don't like the "forced path" feeling when doing this inside a script or utility. I have also considered to make sysinst replace itself by the upgraded newer sysinst during step (2), passing over all state collected so far so it is not very noticable by the user - but we did not have really critical changes in sysinst for the post-binary-extraction phase recently (one might be upcoming that deals with the entroyp setup, I will reconsider this idea then). Martin
Re: Checking whether an rc.d script was invoked from rc
On Tue, Apr 12, 2022 at 12:53:00PM +0200, Edgar Fuß wrote: > > Make the script require CRITLOCALMOUNTED (in -current, or mountcritlocal > > in older versions)? > I don't get that. That wouldn't mount /usr, no? Duh, right - I have /usr in critical_filesystems_local on all machines where it is a separate partition. Martin
Re: Checking whether an rc.d script was invoked from rc
On Tue, Apr 12, 2022 at 12:09:40PM +0200, Edgar Fuß wrote: > Alternatively, I could copy na6 to /local/bin (or whatever). It only needs > libc, libpcap and libm, which are all in /lib, but ldd say the binary > references /usr/lib. How do I change that? Make the script require CRITLOCALMOUNTED (in -current, or mountcritlocal in older versions)? Martin
Re: ZFS - mounting filesystems (Was CVS commit: src/etc)
On Thu, Mar 17, 2022 at 12:52:03PM +, Taylor R Campbell wrote: > Does anyone actually do this -- have local mounts on top of remote > mounts? I do, but all machines affected have / on NFS which makes the setup trivial. Martin
Re: inetd tests failing
On Thu, Mar 17, 2022 at 04:58:56PM +1030, Brett Lymn wrote: > Perhaps a shim program that manages a socket between it and the > underlying daemon - the shim can talk to inetd to coordinate the handoff > of an incoming connection and also being put back onto the idle pool > when the connection closes. My point was: it is pretty simple and efficient to do in setups like httpd where the child and parent are the same and also can cooperate in special ways (like the preford children having a socketpair open to the parent and get their real stdin passed later on wakeup). But in general cases like inetd and child programs not specially written for this setup, it is impossible (or needs special tricks that will kill all the benefits like a shim program). The prefork idea is to have children ready for service as quickly as possible after a new connection has been established - avoiding delay by the OS for creating a new process and initialization time for execing the new servicing binary (and its internal initial initialization time). I don't see how that could be done in inetd context without adding special new kernel support. E.g. we could create a posix_spawn() flag that makes the child process stop immediately after loading and create a new syscall that (if called by the parent) replaces a set of filedescriptors for procs in this state and unblocks them. But maybe I am missing something or misunderstanding the optimization. Martin
Re: inetd tests failing
On Wed, Mar 16, 2022 at 07:53:09AM +1030, Brett Lymn wrote: > They wait on stdin, they don't need to signal done. How does inetd replace their stdin once the connection is established? Many deamons will do socket operations (e.g. querying the peer) early on startup - that would fail (or deliver bogus data) in the pre-fork case. Martin
Re: ZFS - mounting filesystems (Was CVS commit: src/etc)
On Tue, Mar 15, 2022 at 08:30:11AM -0400, Greg Troxel wrote: > I still don't understand and object to this "zfs is special" notion. It is special because it just does not use /etc/fstab (usually). I don't like this part either, but we probably don't want to make our ZFS different from others (or force "legacy" markers). Martin
Re: crypt_r()?
On Wed, Feb 16, 2022 at 10:27:08AM -0500, Mouse wrote: > That sounds like a recipe for disaster. It is a complete fail for > heterogenous environments where the same hash needs to be checkable on > widely disparate hardware, where a small machine may not have the > resources to perform the check _at all_. It is managable and configurable. You can configure your passwd.master to not use argon2 at all, or you can use the settings on one of the smaller machines and copy that over to the others (or only ever add new users on that machine and copy the hash over). We tested the NetBSD passwd implementation using argon2 on a SparcStation LX and it auto-configured itself fine. I should have tested on the VAX and the mac68k too, but updating those was inconveninent at the time nia did the changes - but I'm sure both will work fine too. Martin
Re: installboot: Old BPB too big, use -f (may invalidate filesystem)
On Mon, Feb 14, 2022 at 09:20:45PM +0100, Edgar Fuß wrote: > > If you are trying to setup the machine for NetBSD only use > It's NetBSD only. > > > - if you are UEFI booting from a FAT partition > No, it's plain old BIOS boot (the server is from 2005). > > > - if you really want ffsv1 boot code, then sd0e better not be a FAT file > >system > sd0e is fstype RAID. Maybe installboot should learn about that type of setup ... I guess using -f in your case can do no harm then. Martin
Re: installboot: Old BPB too big, use -f (may invalidate filesystem)
On Mon, Feb 14, 2022 at 07:25:37PM +0100, Edgar Fuß wrote: > What does > installboot: Old BPB too big, use -f (may invalidate filesystem) > mean? > > I have a RAIDframe level 1 RAID consisting of /dev/sd0e and /dev/sd1e. > Now, sd0 failed. I replaced the disc, fdisk'ed and disklabel'ed it and > performed a raidctl -R, which succeeded. Now, I need to write the boot > code, so I can boot from the new disc. But installboot /dev/rsd0e > /usr/mdec/bootxx_ffsv1 gives me above error message. > I tried installboot -n /dev/rsd1e /usr/mdec/bootxx_ffsv1, which works. BPB = a special block in a FAT (or NTFS) file system [BIOS Parameter Block], often within the first 8k of the partition so it may collide with the primary boot code installboot is trying to write there. Installboot on x86 has special checks to avoid destructing such a file system when adding our primary boot code on systems that dual boot into NetBSD and windows. If you are trying to setup the machine for NetBSD only use, you don't really have to care unless you are booting from a FAT file system. But in usual setups this should not happen and something else may be wrong: - if you are UEFI booting from a FAT partition, you do not need any boot blocks installed (don't do the installboot at all), you just need to copy /usr/mdec/*.efi to /EFI/BOOT/ on the FAT partition. - if you really want ffsv1 boot code, then sd0e better not be a FAT file system Martin
Re: brandelf(1)
On Mon, Jan 17, 2022 at 12:28:51PM +0100, Manuel Bouyer wrote: > Hello, > so, to be able to run linux binaries with don't have the Linux type > in its ELF header, I have ported FreeBSD's brandelf(1): > https://www.unix.com/man-page/freebsd/1/brandelf/ > brandelf -- mark an ELF binary for a specific ABI > > Should it go in base, or pkgsrc (or nowhere, it's not usefull enough) ? > It's very small - a single C file of 216 lines (including comments), the > binary is 9.1K. Since it is such a niche tool, I would add it to pkgsrc (but base doesn't hurt much either). Martin
Re: stack overflow in getaddrinfo(3) with a small-sized stack in pthreads
On Sat, Dec 04, 2021 at 11:20:26AM +0100, Havard Eidnes wrote: > > Joerg Sonnenberger wrote in > > : > > |On Mon, Nov 29, 2021 at 08:38:35PM +0700, Robert Elz wrote: > > |> DNS queries (via UDP) are limited to max 512, as that is what the > > |> protocol always required, so can be handled by everything (or should > > be). > > That disregards EDNS0, which in the DNS "in general" is pretty > much universally supported these days. After all, it's nearly 20 > years since it was standardized IIRC. Robert talks about queries, you talk about responses - or am I missing something? They do not share this size limit. Martin
Re: possible bug in fseek of buffered files shared between processes (example uses stdin)
On Tue, Nov 02, 2021 at 10:53:55AM -0700, Carlo Arenas wrote: > Note that the failure in this case can be "fixed" by instead calling > lseek directly, removing the buffer or skipping this early return in > src/lib/libc/stdio/fseeko.c (lines 210 to 228), and that I am showing > below (might be mangled by copy and paste): I am not sure how you conclude this would violate Posix or the C standard. The behaviour inside the single program is correct, and I don't see any guarantees about shared files and current file position usage when using buffered stdio - but maybe I am overlooking something. Martin
Re: openssl 3
On Thu, Sep 30, 2021 at 09:18:23AM -0400, Greg Troxel wrote: > The real question is what kind of pace of update is best, as maintained > upstreams are going to make releases that work with openssl3, and not > being first makes life easier. > > I suspect in a few weeks we'll have a better idea. I am trying hard to make a branch for netbsd-10 actually possible in the very near future. We have only few remaining hard blockers. > > We can't do that. Instead, at some point (probably a bit past the two > > years 1.1 will still receive updates) we will have to bite and switch > > netbsd-9 over to pkgsrc based openssl. > > I don't know what you mean exactly. Certainly at some point pkgsrc > builds on 9 will use pkgsrc openssl. Perhaps long before that. But I > don't see how e.g. postfix in base is going to use pkgsrc openssl. Right, so the admin of an affected machine would have to choose: - update to newer netbsd - update to use pkgsrc postfix - live with outdated openssl Martin
Re: openssl 3
On Thu, Sep 30, 2021 at 08:44:22AM -0400, Greg Troxel wrote: > What are people thinking about > > updating openssl to 3.0.0 in current Yes, someone(tm) should do that! Early to catch fallout quickly, but we'd need commitment from the pkgsrc team to make pkgsrc usefull with that. I don't think we need to sequence this, thing will follow naturally from people hitting more fallout. > pulling up openssl 3 to 9? We can't do that. Instead, at some point (probably a bit past the two years 1.1 will still receive updates) we will have to bite and switch netbsd-9 over to pkgsrc based openssl. It is a pitty that security projects like openssl don't care for ABI compatibility. Martin
Re: [PATCH 1/3] gcc: Add missing parentheses around ternary operator
On Sun, Aug 29, 2021 at 01:42:22PM +1000, Damien Zammit wrote: > --- a/external/gpl3/gcc/dist/libcpp/files.c > +++ b/external/gpl3/gcc/dist/libcpp/files.c > @@ -233,7 +233,7 @@ open_file (_cpp_file *file) > } >else > file->fd = open (file->path, O_RDONLY | O_NOCTTY | O_BINARY > - | (cpp_restricted != NULL) ? O_NONBLOCK : 0, 0666); > + | ((cpp_restricted != NULL) ? O_NONBLOCK : 0), 0666); This is something we would prfer to go via upstream gcc, has it been reported there? If we bring in a temporary local fix, we usually note a reference to the upstrem commit or bug report. Martin
Re: Fwd: Release Maintenance Policy
On Sun, Aug 08, 2021 at 09:16:08AM -0400, matthew sporleder wrote: > FYI we need to pull up a newer version of OpenLDAP More important: we need a new version in HEAD before we branch for netbsd-10 (and it better be 2.6 already). We can not pull up a new release if a major bump of the library is needed. Is there some documenation of the changes? Did some existing functions get removed or change arguments? Martin
Re: possible opendir bug?
On Sun, Jul 11, 2021 at 12:47:53AM -0400, Mouse wrote: > I'm not sure to what extent use of uninitialized memory is considered a > bug when, as here, the code is correct regardless of what value it > contains. It is a bug (and should be detected in builds with proper -fsanitize= compiler options in -current at least). Do you have a reproducer handy? Martin
Re: Strange pthread_self() return value
On Fri, Jun 11, 2021 at 04:41:51PM +, nia wrote: > However, this value comes from the return value of pthread_self(): You did not link with -pthread (so you get the libc stub of pthread_self()). Martin
Re: Adding ?
On Thu, May 13, 2021 at 12:51:45PM +0200, Jaromír Dole?ek wrote: > On the contrary, Linux explicitely #undefs alloca(), then > #define alloca() as __builtin_alloca(), so it always works, regardless > which -std. But isn't that obviously wrong and our version the correct one? (Well, maybe we should never provide anything if there is no __builtin_alloca()) Martin
Re: text layout in sysinst
On Fri, Feb 05, 2021 at 09:12:32PM +0100, Roland Illig wrote: > When I start sysinst from an installed system though, the language > selection is skipped completely (which I didn't understand from reading > the code) This means the message catalog files have not been found - I thought I fixed that sometime back, but I can reproduce the issue in -current. Will have a look. Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Sat, Jan 16, 2021 at 01:21:21PM +0100, Kamil Rytarowski wrote: > Can we overload the ENOSYS return value and return it for CPUs without > hardware assisted random number generator? You mean: with not enough entropy available? The availablility of a hardware random number generator does not really matter in all working setups, no matter what the hardware provides. Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Fri, Jan 15, 2021 at 03:04:21PM +0100, Reinoud Zandijk wrote: > Can we create a random stream for all such situations instead of a file? Say > open a TCP/IP connection to some NetBSD server that spits out a X number of > random bytes by its random generator and then closes the stream? You want this to happen on the *local* network. Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Fri, Jan 15, 2021 at 12:12:05AM +, Taylor R Campbell wrote: > > On Mon, Jan 11, 2021 at 01:25:36AM +, Taylor R Campbell wrote: > > > We might also do something similar with the motd -- add a single line, > > > citing entropy(7) for more details, if there's not enough entropy. > > > > Please don't - that is one of the least usefull places to put such a > > note. > > Can you expand on why? The user reading it there is often not the admin who is able to deal with the situation (and fix it). Many ways of accessing a system do not display /etc/motd. > > I still think that this should be dealt with (once and for all) at > > installation time (as we did for a short period, for some machines and > > install methods) - but apparently it is impossible to reach consensus > > on the wording and supported methods, so I won't touch it. > > It's fine to put _optional_ functionality into sysinst, perhaps in the > utility menu or in the post-installation config menu alongside setting > the timezone and enabling ssh What's not fine is making the user > feel trapped until they take some remedial action about entropy, > before they can proceed to anything else in the installation. This is on a fine line between easy and broken. The code already did display the option only if entropy was in bad shape. Assuming we first try to run the audiorng tool and then re-check entropy state, it would happen even less often (~never on x86, but some arm SoC still come w/o hwrng nor audio - and of course many legacy machines). In principle I agree with you that the best options are optional and can be done later, but in this case things *are* different: - We must be prepared to live with setups where users overlooked the option and just went on (with your recent changes that could work, previously it was impossible - but it still has serious implications). - Changes like root's shell or the default timezone have no long standing impact on the system, but in this case we likely will generate ssh host keys on next boot. How do we force those to be regenerated when entropy is only fixed later? Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Fri, Jan 15, 2021 at 09:25:41AM +0100, Manuel Bouyer wrote: > > If a machine doesn't have any unpredictable inputs, well, there's no > > magic we can do -- you can copy a seed over from another machine (on a > > Yes. I just want to make sure this will remain an option. > For this to work the machine has to come up multiuser with working > network ... The (currently disabled) code in sysinst offers to fetch it via network just like it fetches sets. Martin
Re: tolower()/islower() and char
On Thu, Jan 14, 2021 at 12:28:57PM +0100, Manuel Bouyer wrote: > Does the standard explicitely state that the value should either be > EOF or >= 0 ? Yes, Section 7.4 paragrah 1 second sentence (ISO C 2018): > In all cases the argument is an int, the value of which shall be > representable as an unsigned char or shall equal the value of the > macro EOF. If the argument has any other value, the behavior is > undefined. Martin
Re: tolower()/islower() and char
On Thu, Jan 14, 2021 at 12:14:28PM +0100, Manuel Bouyer wrote: > Any comment about this ? I'm not familiar with these details ... man ctype and search for CAVEATS Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Thu, Jan 14, 2021 at 10:15:41AM +, nia wrote: > I don't think more options should be added to the installer (it has > too many things that are confusing to a new user already), This is the thinking that gets us to fully automated no choice tools like the auto resizing usb images that I find extremely annoying for *my* use cases but realize they are probably good in general. > I also don't think the installer is a place where anything can > be solved "once and for all" I did not say that. But when we are installing via sysinst, it is a good chance to make sure we do not end up with unsafe systems (and if we do handle it there, it never needs to be dealt with later again). Especially when the fix is very easy (and could be auto-running the audiorng tool if audio recording is available, but still needs to deal with machines where there is none, as is the case for many oldish server type machines). Something else(tm) has to be done about cloning VMs and USB images, and maybe it is only documentation that is required. Martin
Re: Waiting for Randot (or: nia and maya were right and I was wrong)
On Mon, Jan 11, 2021 at 01:25:36AM +, Taylor R Campbell wrote: > We might also do something similar with the motd -- add a single line, > citing entropy(7) for more details, if there's not enough entropy. Please don't - that is one of the least usefull places to put such a note. I still think that this should be dealt with (once and for all) at installation time (as we did for a short period, for some machines and install methods) - but apparently it is impossible to reach consensus on the wording and supported methods, so I won't touch it. Martin
Re: vndconfig vs mount_cd9660
On Tue, Dec 29, 2020 at 11:35:15AM +0100, Thomas Klausner wrote: > That did the trick: > > # vnconfig vnd0 file.iso 2048/1/1/1 > # mount_udf -o ro /dev/vnd0a /mnt > > > > disklabel was not more usable than before. > > Is "2048/1/1/1" reasonable or should different values be used? Technically not, but I guess everything but the byte per sector value are unused anyway. I would prefer if vnconfig had a simple sector size arg and could calculate a "reasonable" geometry itself - or even better if mount_udf had an option to force a sector size (instead of asking the disk). Martin
Re: vndconfig vs mount_cd9660
On Mon, Dec 28, 2020 at 10:43:10PM +0100, Thomas Klausner wrote: > I wanted to mount an ISO image (on 9.99.77/amd64), so I did, following the > guide: > > # vnconfig -c vnd0 file.iso > # mount -t cd9660 /dev/vnd0a /mnt > mount_cd9660: /dev/vnd0a on /mnt: Invalid argument May this image contain a UDF file system (instead of ISO9660)? In that case you need to vnconfig it with a geometry specification, so the vnd gets 2048 byte blocks, and then use mount_udf instead of mount_cd9660 (I typically use /dev/vnd0d for that, unless something very special is going on or it is a Sun image with explicit disklabel). Martin
Re: wsmoused broken since 9.1
On Sat, Dec 26, 2020 at 08:49:22AM -0500, Mouse wrote: > > I have to admit I only ever use it to shut up stupid USB mice from > > reconnecting every minute on setups where I boot with serial console > > How does it affect that? There's a 9.1 system at work where (certain) > USB mice will randomly disconnect and reconnect; I never even > considered wsmoused because (among other reasons) I would not expect it > to make any difference. I take it from what you said that it does, but > I do wonder how. Simply by opening the mouse device (and the USB endpoint), the device thinks it is supported and stops the stupid "damn, the OS did miss me attaching and the user won't be able to do anything w/o click and point device, so let us be clever and fix this for them". The way the devices "fix" it is by simulating a disconnect event and then are (re-)discovered by the usb hub (after port and thus device reset). All modern USB mice I have do this. You recognize it by a very stable attach-disconnect interval. Using wsmoused to stop this is certainly overkill, but hey, it was there and intended to talk to the mouse so it doesn't feel lonely. There is a second kind of breakage affecting mice that you did use for some time already and where the cable becomes bad at the entry into the mouse housing (probably due to bending during movements). This makes power for the chip inside the mouse flaky and it randomly disconnects. This can often be fixed by cutting the cable a few cm before entry to the housing, opening the mouse, soldering the internal connector to the now lose end and reassembling. Martin
Re: wsmoused broken since 9.1
On Sat, Dec 26, 2020 at 08:50:22AM +, RVP wrote: > I'm surprised that nobody's noticed this breakage since Aug 2007. > Is console + wsmoused so little used? ;) I have to admit I only ever use it to shut up stupid USB mice from reconnecting every minute on setups where I boot with serial console - so I start the daemon but actually never use it. Martin
Re: wsmoused broken since 9.1
On Wed, Dec 23, 2020 at 06:56:31AM +, RVP wrote: > /usr/sbin/wsmoused has been non-working since slightly > before 9.1 was released. Specifically, any button click > or release generates: Can you narrow down the pullup that broke it? Is it still broken in -current? I see nothing obvious in CHANGES-9.1. Martin
Re: Summary of man-page formatting
On Sun, Nov 15, 2020 at 03:10:08AM +0100, Kamil Rytarowski wrote: > 1. NetBSD base ships with two programs that can format manual pages from > base and most 3rd party software: BSD mandoc (newest) and GPLv2 groff > 1.19.2 (old, from 2005). Ignoring pkgsrc and most parts of what you listed in the part below that I did not quote - for a base release build we currently do need groff as a tool because the release documentation is formatted with it. Before we can even think about removing groff from base, this would have to change. It would probably be relatively easy to switch over the expanding of variables and the conditional parts to some awk script and do the formatting with mandoc, but as you mentioned mandoc is not really meant for such general texts and I don't know what the output would look like. It doesn't need to be a 1:1 replacement though, as long as the end result is nice and readable in the various formats we want. Martin
Re: 9.1: nested functions crash
On Fri, Oct 23, 2020 at 10:56:50AM -0400, Mouse wrote: > Presumably this is NetBSD trying to break execute-out-of-the-stack > malware. This is not an unreasonable default, but, for this use case, > it's a crippling misfeature. How do I disable it? sysctl -a output > contains nothing promising (I looked for "exec" and "stack"); neither > does the usr/src/sys/arch/amd64/conf/GENERIC (same searches). Have a look at paxctl(8). Martin
Re: recent changes to pthread_fork.c:fork() cause static linking to fail if the app provides its own malloc()
On Tue, Jul 14, 2020 at 02:49:00AM +0200, Joerg Sonnenberger wrote: > Replacing malloc is just as invalid from a strict standard compliance > perspective, so *shrug* Why is that? We have e.g. shells/standalone-tcsh that does it. Is it broken now? Martin
Re: Weekday abbreviations in output of cal(1)
On Tue, Jun 30, 2020 at 11:24:51AM +0200, Michael Siegel wrote:
> Well, how would you use date(1) to return the number of days in any
> given month, for example?
Good example, slightly complex but still close to cal|wc ;-)
Martin
--8<--
#! /bin/sh
YEAR=2020
MONTH=6
if [ $MONTH -lt 12 ]; then
next_year=$YEAR
next_month=$(( $MONTH + 1 ))
else
next_year=$(( $YEAR + 1 ))
next_month=1
fi
ms=$( date -d "${YEAR}-${MONTH}-1" '+%s' )
ns=$( date -d "${next_year}-${next_month}-1" '+%s' )
diff=$(( $ns - $ms ))
days=$(( $diff / 86400 ))
echo "${YEAR}-${MONTH} has ${days} days"
Re: Weekday abbreviations in output of cal(1)
On Tue, Jun 30, 2020 at 10:53:38AM +0200, Michael Siegel wrote: > Also, to the best of my knowledge, parsing the output of cal(1) is about > the only method to get certain calendar information in Unix shell > scripts. Now that makes me curious - can you give an example? I think almost always "date +format" is a lot easier, and at quick thought there was no information I could think of that cal(1) gives that would not be available that way (plus maybe some shell arithmetic expressions). Martin
Re: mountcritlocal rcorder (Was: Restrict mtree to specific directories)
I implemented the split and it works nicely for me. I can now just add things like "populate_var" into rc.d which REQUIRE: mountcritlocal and have BEFORE: MOUNTCRITLOCAL to fill the /var tmpfs that has just been mounted. While there I found that random_seed play special tricks and we might consider moving it in this new section as well, or add another stage in between, like: mountcritlocal -> [various] -> MOUNTCRITLOCAL1 -> random_seed -> MOUNTCRITLOCAL Martin Index: distrib/sets/lists/etc/mi === RCS file: /cvsroot/src/distrib/sets/lists/etc/mi,v retrieving revision 1.259 diff -u -p -r1.259 mi --- distrib/sets/lists/etc/mi 15 Sep 2019 19:38:08 - 1.259 +++ distrib/sets/lists/etc/mi 7 Jun 2020 19:21:19 - @@ -200,6 +200,7 @@ ./etc/rc.d/btuartd etc-obsoleteobsolete ./etc/rc.d/ccd etc-sys-rc ./etc/rc.d/cgd etc-sys-rc +./etc/rc.d/clearcritlocal etc-sys-rc ./etc/rc.d/cleartmpetc-sys-rc ./etc/rc.d/cronetc-cron-rc ./etc/rc.d/devpubd etc-sys-rc @@ -242,6 +243,7 @@ ./etc/rc.d/lkm1etc-obsolete obsolete ./etc/rc.d/lkm2etc-obsolete obsolete ./etc/rc.d/lkm3etc-obsolete obsolete +./etc/rc.d/llvmlockdir etc-sys-rc ./etc/rc.d/local etc-sys-rc ./etc/rc.d/lpd etc-lpr-rc ./etc/rc.d/lvm etc-sys-rc @@ -253,6 +255,7 @@ ./etc/rc.d/motdetc-sys-rc ./etc/rc.d/mountalletc-sys-rc ./etc/rc.d/mountcritlocal etc-sys-rc +./etc/rc.d/MOUNTCRITLOCAL etc-sys-rc ./etc/rc.d/mountcritremote etc-sys-rc ./etc/rc.d/mountd etc-nfsserver-rc ./etc/rc.d/moused etc-sysutil-rc Index: etc/rc.d/MOUNTCRITLOCAL === RCS file: etc/rc.d/MOUNTCRITLOCAL diff -N etc/rc.d/MOUNTCRITLOCAL --- /dev/null 1 Jan 1970 00:00:00 - +++ etc/rc.d/MOUNTCRITLOCAL 7 Jun 2020 19:21:19 - @@ -0,0 +1,10 @@ +#!/bin/sh +# +# $NetBSD: NETWORKING,v 1.2 2002/03/22 04:33:57 thorpej Exp $ +# + +# REQUIRE: mountcritlocal +# PROVIDE: MOUNTCRITLOCAL + +# This is a dummy dependency, for services which require local +# disks to be mounted and fully setup Index: etc/rc.d/Makefile === RCS file: /cvsroot/src/etc/rc.d/Makefile,v retrieving revision 1.106 diff -u -p -r1.106 Makefile --- etc/rc.d/Makefile 18 Sep 2019 05:16:15 - 1.106 +++ etc/rc.d/Makefile 7 Jun 2020 19:21:19 - @@ -17,7 +17,7 @@ CONFIGFILES=\ DAEMON DISKS LOGIN NETWORKING SERVERS \ accounting altqd amd apmd automount automountd autounmountd \ bluetooth bootconf.sh bootparams \ - ccd cgd cleartmp cron \ + ccd cgd clearcritlocal cleartmp cron \ devpubd dhcpcd dhcpd dhcpd6 dhcrelay dmesg \ downinterfaces \ envsys \ @@ -27,9 +27,10 @@ CONFIGFILES=\ identd ifwatchd inetd ip6addrctl ipfilter ipfs ipmon ipnat \ ipsec irdaattach iscsi_target iscsid isibootd \ kdc \ - ldconfig ldpd local lpd lvm \ + ldconfig ldpd llvmlockdir local lpd lvm \ makemandb mdnsd mixerctl modules mopd motd mountall \ - mountcritlocal mountcritremote mountd moused mrouted \ + mountcritlocal MOUNTCRITLOCAL \ + mountcritremote mountd moused mrouted \ named ndbootd network newsyslog nfsd nfslocking npf npf_boot \ npfd ntpd ntpdate \ perusertmp pf pf_boot pflogd postfix powerd ppp pwcheck \ Index: etc/rc.d/bootconf.sh === RCS file: /cvsroot/src/etc/rc.d/bootconf.sh,v retrieving revision 1.15 diff -u -p -r1.15 bootconf.sh --- etc/rc.d/bootconf.sh31 Dec 2012 23:21:27 - 1.15 +++ etc/rc.d/bootconf.sh7 Jun 2020 19:21:19 - @@ -4,7 +4,7 @@ # # PROVIDE: bootconf -# REQUIRE: mountcritlocal +# REQUIRE: MOUNTCRITLOCAL # KEYWORD: interactive $_rc_subr_loaded . /etc/rc.subr Index: etc/rc.d/clearcritlocal === RCS file: etc/rc.d/clearcritlocal diff -N etc/rc.d/clearcritlocal --- /dev/null 1 Jan 1970 00:00:00 - +++ etc/rc.d/clearcritlocal 7 Jun 2020
mountcritlocal rcorder (Was: Restrict mtree to specific directories)
On Thu, Jun 04, 2020 at 05:48:34AM +0700, Robert Elz wrote: > And of course, I should have really read mtree(1) before replying, > so the suggested method would just be > > grep whatever original-spec-file [ | grep otherstuff ] | mtree I ended up not using mtree for this, as it was getting more complex than I initialy thought. I replaced it with a simple tar extraction at the right point. I have slight trouble defining that "right point" properly in rcorder terms though. I think /etc/rc.d/mountcritlocal should be split, into the "real mountcritlocal" stuff and all the cleanup happening directly after, including the llvm lock dir setup, being moved into separate scripts "somehow" - so we could have additional scripts added (like my tar extraction) that get run in between. Something like: create an empty dummy script (doing nothing) as a sentinel that provides "mountcritlocal" and requires "mountcritlocal_mount", move the real part of current mountcritlocal to mountcritlocal_mount, move the llvm and cleanup stuff to separate scripts that all require mountcritlocal_mount and have BEFORE mountcritlocal. Martin
Re: Restrict mtree to specific directories
On Tue, Jun 02, 2020 at 04:14:20PM +0200, Martin Husemann wrote: > So in above example I could create a "onlyfile" containing something like: > > ./var > ./var/* > ./var/*/* > ./var/*/*/* > ./var/*/*/*/* > ./var/*/*/*/*/* > > or so I thought - but (a) it does not work and (b) I don't know how deep some > future mtree spec file will be below var. So this -O onlyfile only works if I create a specfile from a directory hierachy. If something does not match a pattern in "onlyfile" the fts node is marked as FTS_SKIP and the search will not descend down that directory. [which also means only a single entry in the above example should have been needed]. Nothing in mtree matches against the specfile entries, so there is no way to filter those out. Should this be considered a bug? Martin
Restrict mtree to specific directories
I have a small issue with mtree(8), that I am not quite sure how to fix best. I would like to create directory hierachies from mtree spec files, but restrict this to selected hierachies (where the mtree files contain a bit more). Example: mkdir /tmp/test && cd /tmp/test && mtree -f /etc/mtree/special -dqUW Now I get this in /tmp/test: drwxr-xr-x 3 root wheel 48 Jun 2 16:04 dev drwxr-xr-x 13 root wheel 528 Jun 2 16:04 etc drwxr-xr-x 2 root wheel0 Jun 2 16:04 root drwxr-xr-x 2 root wheel0 Jun 2 16:04 sbin drwxr-xr-x 9 root wheel 336 Jun 2 16:04 usr drwxr-xr-x 14 root wheel 576 Jun 2 16:04 var What if I want to restrict the created subdirectories to /tmp/test/var ? Should be easy, there are -O onlypaths and -X exclude-file options to mtree. The files are supposed to contain lines of fnmatch(3) patters, which are evaluated with FNM_PATHNAME option against the to-be-created file/dir. So in above example I could create a "onlyfile" containing something like: ./var ./var/* ./var/*/* ./var/*/*/* ./var/*/*/*/* ./var/*/*/*/*/* or so I thought - but (a) it does not work and (b) I don't know how deep some future mtree spec file will be below var. Am I missing some easy trick? Anything obvious I did wrong (so it does not work but should if I do it right)? I am considering to add a variant of -O that specifies directory prefixes and skips all files outside of one of the specified dirs, so my invocation above would become: mkdir /tmp/test && cd /tmp/test && \ mtree -f /etc/mtree/special -dqUW -D ./var Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Tue, May 26, 2020 at 01:37:41AM +0200, Kamil Rytarowski wrote: > I agree here with Joerg. > > At this point it's good to just add a wrapper as in the proposed patch. > Once we will bump libc major, we can rename the syscall and remove > pollts references. > > Weak alias would still be nicer, but it might be not worth the > complexity of tweaking the syscall stab code generation. > > I think it's good to test t_pollts and t_ppoll as in theory they are > different APIs. Long term t_pollts can be removed, after libc major bump. I don't think it is that complex, see the already existing /* * WSYSCALL(weak,strong) is like RSYSCALL(weak), except that weak is * a weak internal alias for the strong symbol. */ (which is of course used for something totaly different right now, but should work for this case too) and probably all it would take is a special clause in lib/libc/sys/Makefile.inc. I would - rename the syscall - either make an alias for the old name or use a wrapper as suggested (but the other way around) - check if there is a good place for a SYS_... define and add that as an alias for syscall(2) - rename the uses in the test case (just like the syscal has been renamed) - add an entry in the libc TODO file for the mythical bump Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Mon, May 25, 2020 at 03:09:09PM +0530, Apurva Nandan wrote: > I meant that I can't create a __weak_alias ppoll(2) to pollts(2) in libc as > pollts(2) has its definition in sys/sys. But yes the resulting weak alias > symbol works from everywhere. The syscall stabs are generated code, but we have full controll over the generator, so adding a variant with an aliased name should be simple. Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Mon, May 25, 2020 at 11:24:14AM +0200, Joerg Sonnenberger wrote: > On Mon, May 25, 2020 at 10:06:22AM +0200, Martin Husemann wrote: > > On Mon, May 25, 2020 at 12:20:53PM +0530, Apurva Nandan wrote: > > > The problem with __weak_alias is that they only work inside the same C > > > file > > > where the original symbol is defined, and don't work outside that file for > > > that symbol. > > > > That is not true. > > It is. ELF symbols are all attached an actual byte range. There is no > way to create an indirect symbol. We must be talking about different things here. The weak alias has to be defined in the same file as the syscall stub, but the resulting symbol would work from everywhere - right? Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Mon, May 25, 2020 at 10:22:12AM +0200, Kamil Rytarowski wrote: > This is a non-standard extension, but everybody mainstream (except > macosx that lacks it?) settled on the name ppoll. ... which is probably the worst name anyone could come up with. Can you explain what alias issues you have? The weak symbol should resolve against the syscall stub, only problem would be syscall(2), but maybe we could add #define SYS_pollts SYS_ppoll somewhere? Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Mon, May 25, 2020 at 12:20:53PM +0530, Apurva Nandan wrote: > The problem with __weak_alias is that they only work inside the same C file > where the original symbol is defined, and don't work outside that file for > that symbol. That is not true. > I have created _ppoll internal function in libc, so it is added to > namespace.h . But why? It is not used in libc, so no need to play namespace games. > I have just created a macro that replace pollts(2) with > ppoll(2) in already written test t_pollts of ATF (I have also split t_poll > into t_poll and t_pollts). I don't see why any of that would be needed. Martin
Re: Addition of ppoll(2), a wrapper around pollts(2)
On Mon, May 25, 2020 at 04:32:14AM +0530, Apurva Nandan wrote: > Hi all, > I have added ppoll(2) implementation to libc/sys, which is a wrapper around > pollts(2) function (basically, pollts(2) and ppoll(2) are aliases, and > NetBSD has pollts(2)). Why not just add a weak alias for the function? It seems the only difference is the name. Did I overlook something? > This is done to increase the compatibility > with Linux, FreeBSD, OpenBSD, DragonFly and Illumos. The man page diff does not suggest any standards that define this function. If everyone else just picked a different name, shouldn't we just adjust the name and add a weak alias for the old name to keep compatibility (and add a note to remove it on the mythical libc bump)? > Please have a look on the patch attached hereby and commit it to base after > a review. The generated configure changes should probably not be done manually here. You did not add any uses of the function internally in libc, so I wonder if the namespace.h change is correct. If we end up with an alias in the end, no new atf test is needed. Martin
Re: getrandom and getentropy
On Thu, May 14, 2020 at 03:12:13PM +0200, Joerg Sonnenberger wrote: > I'd strongly argue that the only category where it really matters > potentially are long term key generators. I would at the same time > consider creating the ssh host keys as part of sysinst, but that's > already setting up the seed file handling too. Note that with -current and assuming you are not on quite new x86 machines the seed file will record 0 bits of entropy. This is why we talked about alternative/manual ways to get entropy on the installed machine. Of course if you pluged in your nice ualea during installation, all will be fine (which is why I enabled it by default yesterday). Martin P.S.: I have realy quite a lot of (very different) machines here and found four(!) so far that have a TRNG build in (three out of five amd64, one mips [erlite3]).
Re: getrandom and getentropy
On Tue, May 12, 2020 at 10:00:20AM +0300, Andreas Gustafsson wrote: > we have entropy when we don't. Adding more sources could mean > reintroducing some timing based sources after careful analysis, but > also things like having the installer install an initial random seed > on the target machine (and if the installer itself lacks entropy, > asking the poor user to pound on the keyboard until it does). I was thinking about the installer part and wondered if there is a less obstrusive way - like: I know I have machines in my local network (a) that I trust and (b) that I know have good entropy. Could the installer (on request) bring up the network and query some things like time from a time server and entropy from a known good source? The keyboard method would of course still be needed as many users won't have the needed local servers. Martin
Re: PATCH libatomic
On Fri, May 08, 2020 at 10:24:43PM +, m...@netbsd.org wrote: > The indirection only applies to the first call. The magic is within > rtld. You are comparing PLT calls with ifunc (where even normal PLT calls have initial resolution overhead, but very tiny - while ifuncs may have arbitrary first time overhead). Thor is talking about something like the LOCK prefix in x86 asm (that is: real atomic ops). Kamil pointed out that the difference is known to the program at compile time or run time. Problem is that the is-not-lockless case will be rarely tested (if at all). Kamil said we should leave that trouble to the programmers and not make decisions for them. Everyone is kind of right here, it is a matter of balancing "costs" and benefits. What I still not get is why it would be hard (or bad) to just have a pkgsrc version (or two, one gcc one llvm) and what costs that would bring (and I don't buy Kamil's "it is an integrated part of the toolchain" argument). Martin
Re: PATCH libatomic
On Fri, May 08, 2020 at 02:26:45PM +0200, Kamil Rytarowski wrote: > With _Atomic() we can mark any arbitrary struct to have serialized > accesses. As I said, with your attitude we shall remove FPU support (and > softfloat) as they are not async safe, not safe in virtualization for > MMU accesses and more just to mention 2 issues. Now come on, that certainly is apples and oranges. Joerg already agreed to have a "working" libatomic in pkgsrc, so all the lazy programmers can use them and 3rd party apps will be happy. Why do you insist on importing it in base? Martin
Re: PATCH libatomic
On Thu, May 07, 2020 at 05:43:02AM +0200, Kamil Rytarowski wrote:
> I propose the following patch:
>
> http://netbsd.org/~kamil/patch-00250-libatomic.txt
+__inline static int
+__futex(volatile uint32_t *uaddr, int op, int val, const struct timespec
*timeout,
+volatile uint32_t *uaddr2, int val2, int val3)
+{
+return syscall(SYS___futex, uaddr, op, val, timeout, uaddr2,
+val2, val3);
+}
Maybe it is not ready for prime time before we have finalized and published
the futex api?
> Local changes:
[..]
> - fixed __atomic_compare_exchange_##n
What do you mean with that?
Martin
Re: posix_spawnp() and PATH - change code or manual ?
On Wed, May 06, 2020 at 02:51:28PM +0200, Joerg Sonnenberger wrote: > The environment is under the control of the user, the code isn't. Ok, but you can set PATH to : or something. Martin
Re: posix_spawnp() and PATH - change code or manual ?
On Wed, May 06, 2020 at 02:29:07PM +0200, Joerg Sonnenberger wrote: > I'm not sure. There is normally no way to force an empty environment > variable, so if you want to not use PATH, the behavior of the current > code is more useful. But you can then just use posix_spawn() instead. Martin
Re: posix_spawnp() and PATH - change code or manual ?
On Wed, May 06, 2020 at 02:33:41PM +0700, Robert Elz wrote: > Either of those is fine with POSIX - but we should really have one consistent > policy. Changing either the manual or the code to match the other is also > easy. What is not so easy is to decide which of the two (or perhaps > something else, like perhaps using the results from a lookup of > sysctl(user.cs_path)) we should actually do. > > Opinions? Make the code follow the manual, it sounds usefull behaviour. Martin
Re: getrandom and getentropy
On Sat, May 02, 2020 at 12:22:20PM +, m...@netbsd.org wrote: > Having compat shims in libc is as good as having compat shims in syscall > because it isn't possible to share raw syscall code between NetBSD and > other operating systems -- our calling convention is different. I can't parse that - a "compat shim" here would be something like a getrandom() function implemented by doing sysctl kern.arandom. Helps everything expecting the getrandom() API and using libc, does not help go or rust. Not sure how other OSes get involved here. Martin
Re: package build failures on netbsd-8 using nbtar, pax, bsdtar
On Mon, Mar 09, 2020 at 08:10:04PM -0400, Greg Troxel wrote: > The package is /usr/pkgsrc/textproc/py-sphinxcontrib-serializinghtml > which was recently updated to 1.1.4. The 1.1.3 tarball looks fine with > NetBSD's /bin/tar but the 1.1.4 throws lots of: I think this pkg needs: EXTRACT_USING=bsdtar Martin
Re: Expose max_align_t unconditionally
On Mon, Mar 09, 2020 at 08:28:58PM +0100, Micha? Górny wrote: > ATF-C++ forces -std=gnu++98. We could remove that but then we fail > on deprecated auto_ptr use. We should fix that, independently of the issue at hand. Martin
Re: Expose max_align_t unconditionally
On Mon, Mar 09, 2020 at 02:49:23PM -0400, Greg Troxel wrote: > I am not really following the issue, but I don't understand why what the > default C++ sublanguage is matters. It would seem that behavior should > be correct for any --std passed to compilers, per that sublanguages's > specification, and what you get by default is just what happens to you > if you don't pass --std. (And I basically think a build that does not > pass an explicit --std is buggy, but that's a separate issue.) The problem is that this c++ lib does not support compiling with -std= for standards < C++11 (IIUC). This is pretty stupid, but IMHO no big deal. We can - expose max_align_t (bogusly) for older C++ versions and hope that every user of this combination is lucky with other parts of the lib - just not support (i.e. #error) on older C++ standard compiles - wait for Jörg to win the upstream battle and go with the fixed libc++ As a C++ developer in real life I would not expect much lossage from the #error variant - most things that really require older standards will have trouble anyway, and patching the few pkgsrc things that eroneously force some older C++ version (but actually do not care) is not a big deal. But I may be overoptimistic ;-) Michal, what I don't understand: what is the exact failure mode (i.e. what is a minimal sample to trigger the lossage)? Is max_align_t used in some header included everywhere, or just specific rarely used headers? Or is for example our in tree groff version not compilable with that libc++? Martin
Re: Expose max_align_t unconditionally
Stupid question: if that libc+++ is only designed to be C++11 and newer, in what real world scenarios does it require exposing max_align_t? For sure you can easily compile the libc++.so* itself with proper standard flags passed to the compiler, so it must be something else - or I'm missing something. Martin
Re: pam_krb5.c - -Werror=format-overflow
On Thu, Feb 06, 2020 at 05:39:53AM -0500, Santhosh Raju wrote: > Alright the solution would be to 'fix gcc' I suppose? :P It might be a makefile bug passing different options to this build step (unintendely). > Will wait for other opinions (to see if there are any). If not will > add the flag to ignore the warning. A flag to supress the warning is fine, but we might need to make it conditional on gcc version or maybe other flags. Martin
Re: pam_krb5.c - -Werror=format-overflow
On Thu, Feb 06, 2020 at 05:16:05AM -0500, Santhosh Raju wrote: > I am guessing the build cluster does not run with MKLIBCSANITIZER=yes > in the build.sh? No, but why would that make a difference here? If it does, it sounds like a gcc bug. Martin
Re: pam_krb5.c - -Werror=format-overflow
On Thu, Feb 06, 2020 at 05:01:55AM -0500, Santhosh Raju wrote: > --- dependall-pam_krb5 --- > /home/fox/projects/netbsd/obj-wip/destdir.amd64/usr/include/security/pam_modules.h: > In function 'pam_sm_setcred': > /home/fox/projects/netbsd/src-wip/lib/libpam/modules/pam_krb5/pam_krb5.c:489:6: > error: null destination pointer [-Werror=format-overflow=] > sprintf(p, "%d", getpid()); > ^~ > /home/fox/projects/netbsd/src-wip/lib/libpam/modules/pam_krb5/pam_krb5.c:485:6: > error: null destination pointer [-Werror=format-overflow=] > sprintf(p, "%d", pwd->pw_uid); > ^ The diagnostic obviously is totally stupid and should be ignored. The Makefile tries to do so for clang already: CWARNFLAGS.clang= -Wno-format-security Instead of modifying the (innocent) source you should extend this for gcc flags instead. The host gcc is totally irrelevant here, as this is compiled by the in-tree cross gcc. So why is it breaking for you but works on the build cluster? Martin
Re: Solving the syslogd problem
On Wed, Jan 29, 2020 at 11:33:22AM +0100, Joerg Sonnenberger wrote: > On Tue, Jan 28, 2020 at 09:21:23PM +, Roy Marples wrote: > > To fix this, I suggest that we split syslogd into syslogd and > > syslogd-network. > > We could also do a much simpler and more radical decision and stop > splitting / and /usr. Of all the partitioning choices available, it > truely seems to be a pointless legacy from extremely constrained > hardware with a significant cost to maintain. However, we still support a lot of this hardware and often there is no easy upgrade path. I did three installations last year using that split. One of them was an amd64 machine and the size restrictions came from a limited selection of boot devices and other OSes occupying parts of the disk already (with no easy way to move things around arbitrarily). Martin
Re: Leak Sanitizer - how to suppress leaks
On Fri, Sep 13, 2019 at 08:33:31AM +0200, Thomas Klausner wrote: > I'm sorry, I totally do not get it the problem with -- in general -- > writing the code in such a way that it properly frees any allocations > it made. > > I suspect there will be corner cases where it will be really hard, and > we can discuss those separately. But discussing about adding a free() > in ps? (If it was done incorrectly, let's fix it.) > > Please enlighten me. There sometimes is a non-trivial amount of work to do to properly free up everything. This gains us nothing when we know it happens exactly once before exiting. I remember timing a compiler pass where the runtime was cut to slightly about 50% by just avoiding freeing the AST (which was built once for the pass, of course). This obviously was decades before Rust and C++, but nevertheless - sometimes the time saving can be huge. Martin
