Re: Add ENOTDIR as possible error to unveil(2)
On Sat, Sep 16, 2023 at 9:47 AM Ingo Schwarze wrote: > Janne Johansson wrote on Sat, Sep 16, 2023 at 11:49:10AM +0200: > > > In case someone wants the change in a diff format, those 5 seconds > > of work are available here: > > http://c66.it.su.se:8080/obsd/unveil.2.diff > > How come jj@ forgot we want diffs inline? > > > +.It Bq Er ENOTDIR > > +.Fa path > > +points to a file that is not a directory. > > That diff is certainly not OK. > It is misleading because a "path" argument pointing to a file > that is not a directory is actually valid. > > The problem only occurs when you add a trailing slash to the name > of a file that is not a directory, like this: > >$ touch tmp.txt >$ ls -l tmp.txt > -rw-r--r-- 1 schwarze schwarze 0 Sep 16 18:00 tmp.txt >$ ls -l tmp.txt/ > ls: tmp.txt/: Not a directory >$ ls -l tmp.txt/foobar > ls: tmp.txt/foobar: Not a directory > > Looking at sys_unveil in /sys/kern/vfs_syscalls.c, i suspect that > errno actually comes from a lower layer, likely namei(9), rather than > from unveil(2) itself. > Yes, from namei(9)/vfs_lookup(9) and the VOP_LOOKUP() calls they make. Basically all the syscalls that take a path should document the standard set of errnos that paths can prompt: ENOTDIR, ENAMETOOLONG, ENOENT, EACCES, ELOOP, EIO, (and EFAULT I guess). Those in the *at() family should document EBADF, and the fd-related causes of ENOTDIR and EACCES. And then there are additional errors that those calls which may create or modify the target path probably need to document: EISDIR and EROFS, maybe ENXIO or EPERM... > In library manual pages, we occasionally have sentences like this: > > execv() may fail and set errno for any of the errors specified > for the library function execve(2). > > The fgetln() function may also fail and set errno for any of the > errors specified for the routines fflush(3), malloc(3), read(2), > stat(2), or realloc(3). > > But it doesn't look like we do that in system call manuals, and > besides, namei(9) appears to lack the RETURN VALUES section. > Hmm, would it be useful to fill in namei(9)'s RETURN VALUES section with the template of the "so you took a path argument..." errnos that syscall manpages can start from? > So, what is our policy with syscall ERRORS sections? > For standardized syscalls, POSIX probably needs to be taken > into account in addition to our implementation. > But for our very own syscalls? Hm... > > access(2), acct(2), chdir(2), chflags(2), chmod(2), chown(2), chroot(2), > execve(2), getfh(2), ktrace(2), link(2), mkdir(2), mkfifo(2), mknod(2), > mount(2), open(2), pathconf(2), quotactl(2), readlink(2), rename(2), > revoke(2), rmdir(2), stat(2), statfs(2), swapctl(2), symlink(2), > truncate(2), unlink(2), utimes(2) all list ENOTDIR, and i don't see > any right now taking a path argument that don't. > > Do we want the following diff? > Yes, I think we want at least that. ok guenther@ > I did not sort the existing return values in alphabetical order, > > there were two out of order in there, but that is a separate > > commit I guess. > > NetBSD has a policy of sorting ERRORS entries alphabetically, we don't. > > Yours, > Ingo > > > Index: unveil.2 > === > RCS file: /cvs/src/lib/libc/sys/unveil.2,v > retrieving revision 1.22 > diff -u -r1.22 unveil.2 > --- unveil.26 Sep 2021 08:03:08 - 1.22 > +++ unveil.216 Sep 2023 16:38:24 - > @@ -158,6 +158,10 @@ > A directory in > .Fa path > did not exist. > +.It Bq Er ENOTDIR > +A component of the > +.Fa path > +prefix is not a directory. > .It Bq Er EINVAL > An invalid value of > .Fa permissions > >
Re: ps.1/kvm documentation
On Mon, Sep 11, 2023 at 5:29 AM Marc Espie wrote: > On Mon, Sep 11, 2023 at 12:10:17PM +0200, Claudio Jeker wrote: > > On Mon, Sep 11, 2023 at 11:02:00AM +0200, Marc Espie wrote: > > > I was reading through ps.1, which has two slightly different options > > > -H Also display information about kernel visible threads. > > > -k Also display information about kernel threads. > > > > > > It's not at all obvious what the difference between these options > might be. > > > > kernel threads == kthread(9) created threads > > > > Those should have K in STAT and the name is in () like: > > 3141 ?? RK/1 4057:57.90 (idle1) > Fun fact: at the current time, all kernel threads are actually full processes, though this just wastes some memory and cache. Part of my (evil) plan in adding thread names was removing an impediment in changing kthread_create(9) to create just a thread inside process 0. > kernel visible threads == __tfork_thread(3) created threads for userland > > applications. For example: > > > > 43838 556612 ?? IpU 0:01.58 firefox (firefox/Cache2 I/O) > > 43838 415551 ?? IpU 0:00.01 firefox (firefox/Cookie) > > 43838 377915 ?? IpU 0:00.01 firefox (firefox/Worker Launcher) > > > > These threads all share the same PID but have different TID. > > > > I think the "kernel visible" is there to tell that pure userland threads > > can not be reported by ps. I think go routines are such an example. > Claudio's descriptions are correct. How to massage those into the ps(1) manpage is unclear to me. > Any of you guys willing to provide a patch to ps(1) and kvm_getprocs(3) ? > I think I've written a diff for kvm_getprocs(3) at least 3 times and each previous time the way the API makes me grit my teeth has resulted in incomplete rewrites of the API and its callers that I haven't been happy enough with to complete and push through. Enough stalling on documenting the current behavior: how about something like the diff below? Philip Index: lib/libkvm/kvm_getprocs.3 === RCS file: /data/src/openbsd/src/lib/libkvm/kvm_getprocs.3,v retrieving revision 1.21 diff -u -p -r1.21 kvm_getprocs.3 --- lib/libkvm/kvm_getprocs.3 11 Aug 2019 15:48:08 - 1.21 +++ lib/libkvm/kvm_getprocs.3 12 Sep 2023 04:25:32 - @@ -91,6 +91,13 @@ processes with real user ID .Fa arg .El .Pp +In addition, if +.Dv KERN_PROC_SHOW_THREADS +is bitwise ORed into +.Fa op +then additional entries for matched processes' associated threads +are returned. +.Pp Only the first .Fa elemsize bytes of each array entry are returned.
Re: kqueue(2) and close-on-exec
I think that changing the behavior of the existing API in a way that gratuitously increases the differences between BSDs is unwise. IMHO, we should follow NetBSD on this and add kqueue1(), that being obviously consistent with the previous 'add flags argument' extensions: pipe2(), dup3(), and accept4(). (As you know, close-on-fork is imposed because the "take a descriptor number" filters are specified in a way tied to a struct filedesc and there's no obvious extension to the kqueue behavior for sharing it across processes. Oh, and as a result trying to do so would result in use-after-frees and/or assertion failures in the kernel. Using a kqueue after exec() is weird, but not logically undefined in that way.) Philip On Sun, Aug 13, 2023 at 4:55 AM Visa Hankala wrote: > FreeBSD and NetBSD have variants of the kqueue(2) system call that > allow setting the close-on-exec flag on the returned file descriptor. > > In general, I think it is good that the flag can be set atomically > for new descriptors. However, it seems to me that it is almost surely > a mistake if a kqueue descriptor is passed over an exec. > > Instead of adding a new system call, maybe close-on-exec should be > enabled automatically by kqueue(2). Today it feels backwards that > close-on-exec is off by default. > > Note that kqueue cannot be inherited by accident in fork-then-exec > situations because fork(2) closes kqueue descriptors for the child > process. > > Index: sys/kern/kern_event.c > === > RCS file: src/sys/kern/kern_event.c,v > retrieving revision 1.197 > diff -u -p -r1.197 kern_event.c > --- sys/kern/kern_event.c 13 Aug 2023 08:29:28 - 1.197 > +++ sys/kern/kern_event.c 13 Aug 2023 10:42:45 - > @@ -932,7 +932,7 @@ sys_kqueue(struct proc *p, void *v, regi > *retval = fd; > LIST_INSERT_HEAD(>fd_kqlist, kq, kq_next); > kq = NULL; > - fdinsert(fdp, fd, 0, fp); > + fdinsert(fdp, fd, UF_EXCLOSE, fp); > FRELE(fp, p); > out: > fdpunlock(fdp); > Index: lib/libc/sys/kqueue.2 > === > RCS file: src/lib/libc/sys/kqueue.2,v > retrieving revision 1.48 > diff -u -p -r1.48 kqueue.2 > --- lib/libc/sys/kqueue.2 13 Aug 2023 08:29:28 - 1.48 > +++ lib/libc/sys/kqueue.2 13 Aug 2023 10:42:45 - > @@ -74,6 +74,7 @@ on a file descriptor will remove any kev > .Pp > .Fn kqueue > creates a new kernel event queue and returns a descriptor. > +The new descriptor has close-on-exec flag set. > The queue is not inherited by a child created with > .Xr fork 2 . > Similarly, kqueues cannot be passed across UNIX-domain sockets. > >
Re: Stop using direct syscall(2) from perl(1)
On Thu, Jul 20, 2023 at 10:59 PM Theo de Raadt wrote:
> Andrew Hewus Fresh wrote:
>
> > One thing to note, on the advice of miod@, I adjusted the mapping of
> > size_t from u_int to u_long, but that means we no longer recognize
> > getlogin_r. When I asked, miod@ suggested that syscalls.master was
> > confused.
> >
> > $ grep getlogin_r /usr/src/sys/kern/syscalls.master /usr/include/unistd.h
> > /usr/src/sys/kern/syscalls.master:141 STD { int
> sys_getlogin_r(char *namebuf, u_int namelen); }
> > /usr/include/unistd.h:intgetlogin_r(char *, size_t)
>
> That's quite a surprise.
>
> syscalls.master will need to be changed, which will change
> struct sys_getlogin_r_args, which is used inside sys_getlogin_r() in
> kern_prot.c. Don't get fooled by the advisory /* */ block you see
> there, it is using the generated struct. But the offset to load the
> field isn't changing, and luckily the incorrect field is smaller than
> the correct field, so I *think* there is no major ABI crank needed.
>
Yeah, at least for syscall args a u_int and a size_t both get passed in a
register-sized space on all our platforms' ABIs, so sys_getlogin_r_args
doesn't actually change size or offsets.
(If it was off_t or long long vs an int or long-sized arg, that would be
different, of course, on 32bit archs, but this change is in the clear.)
It does mean the syscall is implicitly reducing the len argument mod 2^32
on ILP32 archs, so it may fail with ERANGE if passed a 4GB buffer when it
should succeed, but I don't think that merits any sort of syspatch or such.
Philip Guenther
Re: remove extra parentheses
On Tue, Jul 11, 2023 at 9:21 PM Masato Asou wrote: > ok ? > ok guenther@ (I think this was simply from how the original example was simplified in rev 1.25) Philip Guenther
Re: OpenBSD::MkTemp vs Devel::Cover
Yeah, I don't really get what's going on here that Devel::Cover is unhappy
about.
Maybe it's something about how my mkstemps_real() implementation creates
the filehandle that it returns. I see perlxstut(1) now talks about
{Input,InOut,Output}Stream and PerlIO* in the typemap, so maybe I should
change it to skip the newGVgen()+sv_bless() dance, switch from PPCODE: to
CODE: with a return type of InOutStream, and return the PerlIO_fdopen()
result?
(I also suspect it should be using mkostemps(..., O_CLOEXEC), given how
newer perls have switched to setting the close-on-exec flag by default in
the low-level calls, but it's 100% unclear to me what perl's expectations
are for the close-on-exec flag of an fd returned to it by an XS.)
Philip
On Sun, Jul 9, 2023 at 12:30 PM Andrew Hewus Fresh
wrote:
> On Sat, Jul 08, 2023 at 12:09:01PM -0700, Andrew Hewus Fresh wrote:
> > On Sat, Jul 08, 2023 at 11:18:00AM +0200, Marc Espie wrote:
> > > Hey, Philip, you wrote this a long time ago.
> > >
> > > Now, I'm trying to get some coverage out of Devel::Cover on pkg_add,
> > > and somehow, it gets in the way.
> > >
> > > # perl -MDevel::Cover=+select,OpenBSD/.* /usr/sbin/pkg_add random_run
> >
> > > the problem was:
> > >
> > > Found type 9 GLOB(0x64a7292fc10), but it is not representable by the
> Sereal encoding format at
> /usr/local/libdata/perl5/site_perl/amd64-openbsd/Devel/Cover/DB/IO/Sereal.pm
> line 46.
> > >
> > >
> > > I have zero idea if Devel::Cover is to blame or if OpenBSD::MkTemp is
> missing
> > > some magic annotation to cover for _GEN_1, _GEN_2, but the end result
> is
> > > that NO coverage data gets written, none at all (I suspect some missing
> > > annotations, see below)
> >
> > Same, especially when I can't reproduce on my laptop that needs to be
> > updated or on my sparc64 that just got updated today:
>
>
> I take that back, running the same command as root I see the same
> problem.
>
> However, trying to reproduce with a simple case doesn't show me the same
> issue, so not quite sure what Devel::Cover is trying to store.
>
> $ cat mktemp.pl
> #!/usr/bin/perl
> use v5.36;
>
> use OpenBSD::MkTemp qw< mkstemp mkdtemp >;
>
> sub output($fh, $content) {
> print $fh $content;
> }
>
> my $d = mkdtemp("fooX");
> my ($fh, $file) = mkstemp("fooX");
>
> output($fh, "Hello World\n");
>
> close $fh;
>
> $ doas perl -MDevel::Cover=+select,OpenBSD.* ./mktemp.pl
>
> This version of Devel::Cover was built with Perl version 5.036000.
> It is now being run with Perl version 5.036001.
> Attempting to make adjustments, but you may find that some of your modules
> do
> not have coverage data collected. You may need to alter the +-inc,
> +-ignore
> and +-select options.
>
> Devel::Cover 1.40: Collecting coverage data for branch, condition, pod,
> statement, subroutine and time.
> Selecting packages matching:
> OpenBSD.*
> Ignoring packages matching:
> /Devel/Cover[./]
> Ignoring packages in:
> /usr/local/libdata/perl5/site_perl/sparc64-openbsd
> /usr/local/libdata/perl5/site_perl
> /usr/libdata/perl5/sparc64-openbsd
> /usr/libdata/perl5
> Devel::Cover: Deleting old coverage for changed file mktemp.pl
> Devel::Cover: getting BEGIN block coverage
> Devel::Cover: 100% - 0s taken
> Devel::Cover: getting CHECK block coverage
> Devel::Cover: 100% - 0s taken
> Devel::Cover: getting END/INIT block coverage
> Devel::Cover: 100% - 0s taken
> Devel::Cover: getting CV coverage
> Devel::Cover: 100% - 0s taken
> Devel::Cover: Writing coverage database to
> /tmp/x/cover_db/runs/1688930470.15647.20075
> -- -- -- -- -- --
> --
> File stmt bran condsubpod time
> total
> -- -- -- -- -- --
> --
> ...openbsd/OpenBSD/MkTemp.pm 77.7 25.0 25.0 83.30.00.7
> 61.7
> mktemp.pl 100.0n/an/a 100.0n/a 99.3
> 100.0
> Total 87.5 25.0 25.0 88.80.0 100.0
> 74.5
> -- -- -- -- -- --
> --
>
>
Re: syscall(SYS_sysctl) broken in 7.3
On Wed, May 10, 2023 at 1:29 PM Alexander Bluhm wrote: > We use Perl syscall(2) function to implement sysctl(2) system calls. > This is broken since OpenBSD 7.3. A sample program looks like this: > ... > kdump shows two problems: > 1. The mib is not printed correctly. > 2. The final argument newlen should be 0x10, but is 0x8891aa893c5. > > 30762 perl CALL (via syscall) > sysctl(538970683.538976288.538976288.1718165536,0,0,0x8891d2e1de0,0x8891aa893c5) > 30762 perl RET sysctl -1 errno 22 Invalid argument > > 1. The syscall code now contains the KTRC_CODE_SYSCALL flag, but > the kernel checks code == SYS_sysctl. So the mib is not added to > ktrace output. Fix is easy by using KTRC_CODE_MASK. > ok guenther@ on the first chunk, the ktrsyscall() change. The flag is only passed to ktrsyscall(), not ktrsysret(), so the second chunk of your diff is unnecessary. > 2. Was introdced by this commit in libc. > > revision 1.21 > date: 2023/01/11 01:55:17; author: mortimer; state: Exp; lines: +24 > -13; commitid: 72pYktDvmJhq7OyF; > Add retguard to amd64 syscalls. > > Since we got rid of padded syscalls we have enough registers to do this. > > ok deraadt@ ok kettenis@ > > It assumes that syscalls only have 6 parameters, which is not true > if syscall(2) adds the syscall number. Now the final argument > contains retguard values from the stack. > This is why we can't have nice things. > The easies way to fix this without reverting retguard for all > syscalls, is to use revision 1.20 of SYS.h in libc for syscall.S. > Just manually include the old code instead of current SYS.h. > > I am aware that syscall(2) should go way, but currently I need it. > > ok? > Ick. How about this instead, which preserves retguard protection for syscall(2) and doesn't copy all of SYS.h? Index: lib/libc/arch/amd64/sys/syscall.S === retrieving revision 1.8 diff -u -p -r1.8 syscall.S --- lib/libc/arch/amd64/sys/syscall.S 7 May 2016 19:05:21 - 1.8 +++ lib/libc/arch/amd64/sys/syscall.S 11 May 2023 02:13:20 - @@ -39,4 +39,18 @@ #include "SYS.h" -RSYSCALL(syscall) +SYSENTRY(syscall) + RETGUARD_SETUP(_thread_sys_syscall, r11) +#ifdef _RET_PROTECTOR + pushq 8(%rsp) /* repush 6th argument */ +#endif + RETGUARD_PUSH(r11) + SYSTRAP(syscall) + HANDLE_ERRNO; + RETGUARD_POP(r11) +#ifdef _RET_PROTECTOR + addq$8,%rsp /* pop 6th argument */ +#endif + RETGUARD_CHECK(_thread_sys_syscall, r11) + ret +SYSCALL_END(syscall)
Re: converting perl stuff to v5.36
On Sun, May 7, 2023 at 6:13 AM Marc Espie
wrote:
> It is generally a good thing, I'm mostly talking about
> the "signatures" stuff that allows functions with stuff that looks like
> usual languages.
> Other benefits include somewhat "cheap" check for correct number of
> parameters
> and generally making code shorter.
>
Yep. The benefit that the latter is enforced on all calls, including
method calls (where prototypes did *not* apply) is great and caught several
mismatched calls in the source tree I work on at $DAYJOB.
(I also hope that the binding is/will become more efficient than writing
> stuff manually)
>
I think I saw something about "30% faster" for the binding operation(s) (on
some hardware, with some compiler...).
Stuff to watch out for:
>
> - it supersedes the syntax for prototypes. So if you had prototypes
> (I did!), you need to explicitly write :prototypes in front of them.
>
> For instance, constant methods like
> package X;
> sub is_okay()
> {
> 1;
> }
>
> would become:
>
> package X;
> sub is_okay :prototype() ($)
> {
> 1;
> }
>
>
> under the new rules.
> I'm actually wondering whether keeping the prototype is worth it.
>
For methods, the answer has to be "no", since the prototype has no effect
there. We found we had lots of prototypes in our tree which were actually
*wrong* and would have resulted in calls being rejected if prototypes
weren't ignored for methods.
For plain subs, I would only keep them if they _really_ help the calls look
for more perl-ish, by whatever scale you currently measure that. Maybe a
(&@) prototype so you can do "mysub { s/implicit/sub here/ } qw(args here)"
ala map and grep, but...eh.
(Going forward, I probably wouldn't use an empty prototype directly without
thinking and maybe commenting why I _didn't_ do "use constant" instead.)
> - use v5.36; implies "use strict;" "use warnings;" and "use feature
> qw(say);"
> which is cool
>
Not just 'say' but the entire "use feature :5.36;" bundle. After
'signatures', 'state' is my favorite from that set, making one-line lazy
helper-object setup easy, ala:
sub someclass {
return state $someclass //= SomeClass->new({ blah => 'blah' });
}
sub anotherclass {
return state $anotherclass //= AnotherClass->new({ someclass =>
someclass(), other => 'stuff' });
}
(In a class, you would just make it a member and not need 'state', of
course.)
> - parameters which aren't used can use a single $ or something, which
> leaves
> the question of documentation.
>
> After fiddling a bit with it, I've settled on documenting the method
fully in comments, e.g., for instance:
>
Yeah, that's what I settled on too.
- lambdas (anonymous subs) can use signatures. Note that stuff like
> File::Find explicitly takes 0 parameters, whereas stuff like signal handler
> is woefully underspecified !
>
Yeah, the downside of signatures is that by default it makes adding
parameters a breaking change and can thus calcify the interface. Something
for authors of shared modules that have callbacks to carefully consider. :/
Philip Guenther
Re: cleanup vmm_start_vm, simplifying fd cleanup
On Fri, Apr 7, 2023 at 9:44 AM Dave Voutila wrote:
...
> Touch longer, but won't generate ktrace noise by blind calls to close(2)
> and also accounts for the other error conditions (EINTR, EIO).
>
> For EIO, not sure yet how we want to handle it other than log it.
>
> For EINTR, we want to account for that race and make sure we retry since
> the vmm process is long-lived and could inadvertently keep things like
> tty fds or disk image fds open after the guest vm terminates.
>
So, this is an area where
* the current POSIX standard leaves the behavior unspecified
* everyone (except HP-UX) made close() *always* invalidate the fd, even if
it fails with EINTR
* the next version of POSIX requires close() to *not* invalidate the fd if
it fails with EINTR:
* in those cases where something was interrupted but close()
invalidates the fd and previously return EINTR
they are supposed to now return EINPROGRESS
* also, a new posix_close() interface and POSIX_CLOSE_RESTART #define,
yadda yadda,
* OpenBSD has not changed to that behavior or added those interfaces at
this time, AND
* I'm not sure vmd can even hit this issue due to what it'll be closing!
Last things first: when does close() return EINTR? My understanding is
that the only case that happens in OpenBSD is interrupt of a FINAL close
(no other dup()s open, etc) of a file on NFS with unflushed writes. I
think it could happen on other systems on interrupt of close of a
rewinding-tape devices (e.g. /dev/rst0) but I don't actually see how that
would happen in OpenBSD. Are any of the fds in this case subject to those
sorts of concerns? If they're all either duplicates in a forked process,
or file types that can't block in close like that (say, sockets and
/dev/vmd devices or such), then this is all ignorable.
Meanwhile, the loop you're writing is correct for a future that OpenBSD has
not embraced. You could kind of future-proof it by making the loop
conditional on POSIX_CLOSE_RESTART being defined: if it's defined then
presumably close() has the new POSIX behavior, if not (the current state!)
then the loop is useless or actively dangerous, depending on whether the
process is multi-threaded.
I suspect at some point we'll:
* map EINTR to EINPROGRESS in sys_close()
* #define POSIX_CLOSE_RESTART 0 /* explicitly blessed by POSIX */
* add to libc int posix_close(int fd, int flags) { int ret = close(fd);
return flags ? EINVAL : ret; }
'cause it'll mean less patching for ports, but someone will need to check
for anything in tree that tries to be clever about close() returning EINTR.
Overall , a great use of everyone's time. /s
Philip
Re: Disabling MULTICAST flag on an interface. Force outgoing multicast traffic to a specific interface
On Wed, Mar 1, 2023 at 9:58 AM Luca Di Gregorio wrote: > 1) does anyone know if there is a way to disable MULTICAST on a single > interface? > I don't see any option in ifconfig to do this. > There is not. > 2) Can outgoing multicast traffic be routed to a specific interface? I > don't see any option in route. It seems that the first interface, by id, is > chosen. > Unless you enable a multicast routing daemon, it just follows normal IP routing rules and will follow the default route if there's no better match. If you want to force it to a specific interface, just add a route for 224.0.0.0/4 pointing to that interface. (Of course, if you haven't set multicast=YES in /etc/rc.conf.local then /etc/netstart will create one of those routes itself with the 'reject' flag set to block all multicast, but presumably you've already set that correctly.) Philip Guenther
Re: llvm-strip vs ld.bfd (at least on i386): SIGABRT in sys_execve
On Wed, Feb 15, 2023 at 7:48 AM Stuart Henderson wrote: > Noticed while testing with LLVM 15, but it affects plain -current > as well. If I take a binary that was linked with ld.bfd and strip it > (i.e. this is now using llvm-strip), it breaks the output file in > such a way that it cannot be executed: > ... > Seems that the SIGABRT is from kern_exec's sys_execve -> > exec_process_vmcmds() failing. > ... > Does anyone have an idea what might be wrong please? > llvm-strip is somehow ignoring the alignment requirements of the segments. If you look at the "readelf -l" output instead: Good: Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align PHDR 0x34 0x0034 0x0034 0x001e0 0x001e0 R E 0x4 INTERP 0x001000 0x2000 0x2000 0x00013 0x00013 R 0x1 [Requesting program interpreter: /usr/libexec/ld.so] LOAD 0x00 0x 0x 0x0058d 0x0058d R E 0x1000 LOAD 0x001000 0x2000 0x2000 0x003e8 0x003e8 R 0x1000 ... Note: offset == virtaddr mod alignment Bad: Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align PHDR 0x34 0x0034 0x0034 0x001e0 0x001e0 R E 0x4 INTERP 0x00058d 0x2000 0x2000 0x00013 0x00013 R 0x1 [Requesting program interpreter: /usr/libexec/ld.so] LOAD 0x00 0x 0x 0x0058d 0x0058d R E 0x1000 LOAD 0x00058d 0x2000 0x2000 0x003e8 0x003e8 R 0x1000 Boom, that second LOAD does not have offset == virtaddr mod alignment. Now, the sections that go into that segment have a max alignment of 4 and llvm-strip's changes abides by that, but *IF* it's not going to keep the segments page-aligned then it should be adjusting the virtaddr field of the LOAD segment to keep the offset aligned with the virtaddr (and adjusting the sizes so the LOAD continues to cover the total data). Or it shouldn't be screwing with the packing like that. Philip Guenther
Re: vmm: mask WAITPKG cpuid feature to hide TPAUSE
On Sat, Jan 7, 2023 at 11:04 AM Dave Voutila wrote: > Bringing this to tech@ to increase my chance of someone testing my > diff. > > As reported in this thread on misc@ [1], I believe newer Intel hardware > may be experiencing issues hosting Linux guests under vmm/vmd. It looks > like there are some newer instructions Intel added (TPAUSE specifically) > that also involve some new MSR(s). > > I don't have 12th gen Intel hardware to test this on (I think that's > Alder Lake). I'd like to mask this feature from vmm guests since it's > related to an MSR we don't yet pass through or emulate and has to do > with the TSC (which has it's own challenges in vmm). > > For someone testing, you should be able to grab an Alpine Linux iso > (-virt flavor) and boot it with vmd with the diff. (Without it should > "hang" and spike CPU or just die.) Also check that WAITPKG shows up in > your dmesg on the cpu feature output. > This seem like it'll obviously work, but I guess it seems to me that this "opt-out" approach is generally unsafe/unstable and vmd should consider actively switching to "opt-in" on all these CPUID feature bits. I mean, what bits are defined in the SEFF first-leaf EDX that _do_ work with vmd? Philip
Re: add thread names to vmd(8)
On Sun, Jan 8, 2023 at 8:50 AM Dave Voutila wrote: > Now that guenther@ landed thread names, this should help users > understand vmd host cpu utilization better as it lets us distinguish > between the libevent thread and the vcpu thread. > > Example ps output: > > $ ps -AHf | grep vmd > 30584 502984 p2 S+p 0:00.76 | |-- obj/vmd -d > 46975 512305 ?? Sc 0:00.01 | | |-- vmd: priv (vmd) > 39911 487276 ?? Spc 0:00.76 | | |-- vmd: control (vmd) > 58206 418514 ?? Spc 0:00.01 | | |-- vmd: vmm (vmd) > 43525 215149 ?? Sp 0:00.00 | | | |-- vmd: openbsd (vmd/event) > 43525 316255 ?? Rp/5 0:00.77 | | | |-- vmd: openbsd (vmd/vcpu-0) > 43525 270611 ?? Sp 0:00.00 | | | `-- vmd: openbsd (vmd) > 43206 562455 ?? Spc 0:00.01 | | |-- vmd: agentx (vmd) > 70005 431096 p3 R+p/90:00.00 | |-- grep vmd > > ok? > ... > --- usr.sbin/vmd/vm.c > +++ usr.sbin/vmd/vm.c > ... > @@ -1353,6 +1355,10 @@ run_vm(int child_cdrom, int > child_disks[][VM_MAX_BASE_ > __func__, i); > return (ret); > } > + > + memset(tname, 0, sizeof(tname)); > Delete the memset(); the API requires a C string and snprintf() already guarantees that. Other than that, ok guenther!
Re: [trivial patch] remove superflous #ifdef
On Sat, Dec 31, 2022 at 3:41 PM Crystal Kolipe wrote: > The KASSERT macro is a NOP unless DIAGNOSTIC is defined, so it doesn't > need to > be in an #ifdef here: > ok guenther@ Note: you should hold this diff until 2023 in Calgary and then commit, so you can start 2023 with a perfect "negative lines committed" record.
Re: [patch] add show.c style flag descriptions to route(8)
On Thu, Dec 22, 2022 at 12:04 PM Jason McIntyre wrote: > On Thu, Dec 22, 2022 at 11:49:09AM -0800, Philip Guenther wrote: > > On Thu, Dec 22, 2022 at 11:33 AM Jason McIntyre > wrote: > > ... > > > > > i have to say, i'm getting nowhere with this SYNOPSIS: > > > > > > netstat -r [-AFu] [-f address_family] [-M core] [-N system] > > > [-p protocol] [-T rtable] > > > > > > i mean the combination of -r and -p. is it wholly redundant? > > > > > > > Yeah, looking now I think -p is ignored when -r is supplied. I guess I > > missed that when we worked out rev 1.88 of netstat.1 > > > > Philip > > so i'd rather deal with that separately. diff below to remove -p > from the -r line. > > ok? > > jmc > > Index: main.c > === > RCS file: /cvs/src/usr.bin/netstat/main.c,v > retrieving revision 1.123 > diff -u -p -r1.123 main.c > --- main.c 4 Dec 2022 23:50:48 - 1.123 > +++ main.c 22 Dec 2022 20:01:56 - > @@ -485,8 +485,7 @@ usage(void) > " netstat -s [-gru] [-f address_family] [-p protocol]\n" > " netstat -g [-lnu] [-f address_family]\n" > " netstat -R\n" > - " netstat -r [-AFu] [-f address_family] [-M core] [-N > system] [-p protocol]\n" > - " [-T rtable]\n" > + " netstat -r [-AFu] [-f address_family] [-M core] [-N > system] [-T rtable]\n" > " netstat -P pcbaddr [-v] [-M core] [-N system]\n"); > exit(1); > } > Index: netstat.1 > === > RCS file: /cvs/src/usr.bin/netstat/netstat.1,v > retrieving revision 1.96 > diff -u -p -r1.96 netstat.1 > --- netstat.1 22 Dec 2022 07:23:29 - 1.96 > +++ netstat.1 22 Dec 2022 20:01:56 - > @@ -72,7 +72,6 @@ > .Op Fl f Ar address_family > .Op Fl M Ar core > .Op Fl N Ar system > -.Op Fl p Ar protocol > .Op Fl T Ar rtable > .Nm > .Fl P Ar pcbaddr > ok guenther@
Re: [patch] add show.c style flag descriptions to route(8)
On Thu, Dec 22, 2022 at 11:33 AM Jason McIntyre wrote: ... > i have to say, i'm getting nowhere with this SYNOPSIS: > > netstat -r [-AFu] [-f address_family] [-M core] [-N system] > [-p protocol] [-T rtable] > > i mean the combination of -r and -p. is it wholly redundant? > Yeah, looking now I think -p is ignored when -r is supplied. I guess I missed that when we worked out rev 1.88 of netstat.1 Philip
Re: riscv64: drop unused WEAK_REFERENCE macro
On Fri, Dec 2, 2022 at 3:17 PM Jeremie Courreges-Anglas wrote: > > WEAK_REFERENCE seems to come from FreeBSD, it's not used in our tree. > (WEAK_ALIAS is defined a few lines above). > > ok? > ok guenther@
Re: hppa: switch to clockintr(9)
Took a couple tries due to my farting around (and not wanting to leave the power-sucking hppa running when I couldn't monitor progress) but it completed a make kernel, make build, make release cycle with reboots. Ship it? Philip Guenther On Sat, Dec 3, 2022 at 9:26 PM Philip Guenther wrote: > On Sat, Dec 3, 2022 at 5:43 PM Scott Cheloha > wrote: > >> On Sun, Nov 06, 2022 at 07:42:49PM +, Scott Cheloha wrote: >> > This patch switches hppa to clockintr(9). >> > >> > kettenis@ has tested it on his B2000, which is not MP. >> > >> > It probably needs testing on an MP hppa machine. I am uncertain how >> > common these things are. >> > >> > Notes: >> > >> > - hppa machines now have a randomized statclock(). >> > >> > - Rename cpu_hardclock() to itmr_intr() keep the name accurate. >> > It no longer runs just hardclock(). >> >> 1 month bump. >> >> I am still looking for a person with a multiprocessor hppa machine to >> test this patch. >> >> It has been tested successfully on uniprocessor hppa machines (kernel >> builds, kernel boots, survived a release build, survived upgrade from >> the resulting bsd.rd upgrade). >> >> GENERIC.MP testing is the missing piece before this is ready to >> commit. >> > > Have rebooted to a kernel with this and now building userland (w/-j3) on a > (2 cpu) J6700. > Both cpus are showing about as busy as usual in 'top'. Will report back > when it either completes, fails, or blows up. > > Philip Guenther > >
Re: hppa: switch to clockintr(9)
On Sat, Dec 3, 2022 at 5:43 PM Scott Cheloha wrote: > On Sun, Nov 06, 2022 at 07:42:49PM +, Scott Cheloha wrote: > > This patch switches hppa to clockintr(9). > > > > kettenis@ has tested it on his B2000, which is not MP. > > > > It probably needs testing on an MP hppa machine. I am uncertain how > > common these things are. > > > > Notes: > > > > - hppa machines now have a randomized statclock(). > > > > - Rename cpu_hardclock() to itmr_intr() keep the name accurate. > > It no longer runs just hardclock(). > > 1 month bump. > > I am still looking for a person with a multiprocessor hppa machine to > test this patch. > > It has been tested successfully on uniprocessor hppa machines (kernel > builds, kernel boots, survived a release build, survived upgrade from > the resulting bsd.rd upgrade). > > GENERIC.MP testing is the missing piece before this is ready to > commit. > Have rebooted to a kernel with this and now building userland (w/-j3) on a (2 cpu) J6700. Both cpus are showing about as busy as usual in 'top'. Will report back when it either completes, fails, or blows up. Philip Guenther
Re: Use `ps_mtx' mutex(9) to protect `ps_ucred' and unlock setuid(2), ...
On Thu, Dec 1, 2022 at 10:31 AM Vitaliy Makkoveev wrote:
...
> --- sys/sys/sysctl.h7 Nov 2022 14:25:44 - 1.231
> +++ sys/sys/sysctl.h1 Dec 2022 18:15:06 -
> @@ -587,7 +587,7 @@ struct kinfo_vmentry {
>
> #define_FILL_KPROC_MIN(a,b) (((a)<(b))?(a):(b))
>
> -#define FILL_KPROC(kp, copy_str, p, pr, uc, pg, paddr, \
> +#define FILL_KPROC(kp, copy_str, p, pr, pg, paddr, \
> praddr, sess, vm, lim, sa, isthread, show_addresses) \
>
...
> - (kp)->p_svgid = (uc)->cr_svgid; \
> + PR_LOCK(pr);\
> + (kp)->p_uid = (pr)->ps_ucred->cr_uid; \
>
Nope. As the block comment about this notes, FILL_KPROC() is shared
between the kernel and libkvm and takes each structure pointer separately
as, for example, pr->ps_ucred has the kva address, not the address of the
ucred struct that libkvm has separately read into user memory.
Now, you _could_ have libkvm update pr->ps_ucred to point to its user-space
copy. However, that would make ucred handling different from the other
sub-structures of struct proc and MOST of those we need the real kva for
the show_address functionality.
Not sure if this is the yak-shave you want right now...
(libkvm will obviously also need no-op #defines for PR_LOCK() etc)
Philip
Re: sparc64: constify ddb instruction table
On Fri, Oct 21, 2022 at 11:04 AM Klemens Nanni wrote:
> --- a/sys/arch/sparc64/sparc64/db_disasm.c
> +++ b/sys/arch/sparc64/sparc64/db_disasm.c
>
...
> @@ -877,7 +877,7 @@ struct sparc_insn sparc_i[] = {
> vaddr_t
> db_disasm(vaddr_t loc, int altfmt)
> {
> - struct sparc_insn* i_ptr = (struct sparc_insn *)_i;
> + const struct sparc_insn *i_ptr = (const struct sparc_insn
> *)_i;
>
What's with that cast? Is it only there because sparc_i is an array and
it's wrong to take its address when we just want a pointer to its first
element? I mean, shouldn't that line just (with const) be:
const struct sparc_insn *i_ptr = sparc_i;/* or _i[0] */
?
Philip Guenther
Re: Change pru_rcvd() return type to the type of void
ok guenther@
(Thanks!)
On Sat, Sep 10, 2022 at 10:20 AM Vitaliy Makkoveev wrote:
> We have no interest on pru_rcvd() return value. Also, we call pru_rcvd()
> only if the socket's protocol have PR_WANTRCVD flag set. Such sockets
> are route domain, tcp(4) and unix(4) sockets.
>
> This diff keeps the PR_WANTRCVD check. In other hand we could always
> call pru_rcvd() and do "pru_rcvd != NULL" check within, but in the
> future with per buffer locking, we could have some re-locking around
> pru_rcvd() call and I want to do it outside wrapper.
>
>
> Index: sys/kern/uipc_usrreq.c
> ===
> RCS file: /cvs/src/sys/kern/uipc_usrreq.c,v
> retrieving revision 1.185
> diff -u -p -r1.185 uipc_usrreq.c
> --- sys/kern/uipc_usrreq.c 3 Sep 2022 22:43:38 - 1.185
> +++ sys/kern/uipc_usrreq.c 10 Sep 2022 18:51:42 -
> @@ -363,7 +363,7 @@ uipc_shutdown(struct socket *so)
> return (0);
> }
>
> -int
> +void
> uipc_rcvd(struct socket *so)
> {
> struct socket *so2;
> @@ -390,8 +390,6 @@ uipc_rcvd(struct socket *so)
> default:
> panic("uipc 2");
> }
> -
> - return (0);
> }
>
> int
> Index: sys/net/rtsock.c
> ===
> RCS file: /cvs/src/sys/net/rtsock.c,v
> retrieving revision 1.355
> diff -u -p -r1.355 rtsock.c
> --- sys/net/rtsock.c8 Sep 2022 10:22:06 - 1.355
> +++ sys/net/rtsock.c10 Sep 2022 18:51:42 -
> @@ -115,7 +115,7 @@ int route_attach(struct socket *, int);
> introute_detach(struct socket *);
> introute_disconnect(struct socket *);
> introute_shutdown(struct socket *);
> -introute_rcvd(struct socket *);
> +void route_rcvd(struct socket *);
> introute_send(struct socket *, struct mbuf *, struct mbuf *,
> struct mbuf *);
> introute_abort(struct socket *);
> @@ -299,7 +299,7 @@ route_shutdown(struct socket *so)
> return (0);
> }
>
> -int
> +void
> route_rcvd(struct socket *so)
> {
> struct rtpcb *rop = sotortpcb(so);
> @@ -314,8 +314,6 @@ route_rcvd(struct socket *so)
> ((sbspace(rop->rop_socket, >rop_socket->so_rcv) ==
> rop->rop_socket->so_rcv.sb_hiwat)))
> rop->rop_flags &= ~ROUTECB_FLAG_FLUSH;
> -
> - return (0);
> }
>
> int
> Index: sys/netinet/tcp_usrreq.c
> ===
> RCS file: /cvs/src/sys/netinet/tcp_usrreq.c,v
> retrieving revision 1.207
> diff -u -p -r1.207 tcp_usrreq.c
> --- sys/netinet/tcp_usrreq.c3 Sep 2022 22:43:38 - 1.207
> +++ sys/netinet/tcp_usrreq.c10 Sep 2022 18:51:42 -
> @@ -792,18 +792,17 @@ out:
> /*
> * After a receive, possibly send window update to peer.
> */
> -int
> +void
> tcp_rcvd(struct socket *so)
> {
> struct inpcb *inp;
> struct tcpcb *tp;
> - int error;
> short ostate;
>
> soassertlocked(so);
>
> - if ((error = tcp_sogetpcb(so, , )))
> - return (error);
> + if (tcp_sogetpcb(so, , ))
> + return;
>
> if (so->so_options & SO_DEBUG)
> ostate = tp->t_state;
> @@ -820,7 +819,6 @@ tcp_rcvd(struct socket *so)
>
> if (so->so_options & SO_DEBUG)
> tcp_trace(TA_USER, ostate, tp, tp, NULL, PRU_RCVD, 0);
> - return (0);
> }
>
> /*
> Index: sys/netinet/tcp_var.h
> ===
> RCS file: /cvs/src/sys/netinet/tcp_var.h,v
> retrieving revision 1.157
> diff -u -p -r1.157 tcp_var.h
> --- sys/netinet/tcp_var.h 3 Sep 2022 22:43:38 - 1.157
> +++ sys/netinet/tcp_var.h 10 Sep 2022 18:51:42 -
> @@ -725,7 +725,7 @@ int tcp_connect(struct socket *, struct
> int tcp_accept(struct socket *, struct mbuf *);
> int tcp_disconnect(struct socket *);
> int tcp_shutdown(struct socket *);
> -int tcp_rcvd(struct socket *);
> +voidtcp_rcvd(struct socket *);
> int tcp_send(struct socket *, struct mbuf *, struct mbuf *,
> struct mbuf *);
> int tcp_abort(struct socket *);
> Index: sys/sys/protosw.h
> ===
> RCS file: /cvs/src/sys/sys/protosw.h,v
> retrieving revision 1.55
> diff -u -p -r1.55 protosw.h
> --- sys/sys/protosw.h 5 Sep 2022 14:56:09 - 1.55
> +++ sys/sys/protosw.h 10 Sep 2022 18:51:42 -
> @@ -72,7 +72,7 @@ struct pr_usrreqs {
> int (*pru_accept)(struct socket *, struct mbuf *);
> int (*pru_disconnect)(struct socket *);
> int (*pru_shutdown)(struct socket *);
> - int (*pru_rcvd)(struct socket *);
> + void(*pru_rcvd)(struct socket *);
> int (*pru_send)(struct socket *, struct mbuf *, struct mbuf *,
> struct mbuf *);
> int (*pru_abort)(struct socket *);
> @@ -336,12
Re: move PRU_CONTROL request to (*pru_control)()
Yes, splitting {tcp,udp}_usrreqs and {tcp,udp}6_usrreqs is clearly the
Right Thing.
ok guenther@
Unrelated to this specific callback, but I think you should consider
splitting out uipc_dgram_usrreqs out from uipc_usrreqs, as the SOCK_DGRAM
case differs from the other two for multiple callbacks.
Philip
On Thu, Sep 1, 2022 at 10:54 AM Vitaliy Makkoveev wrote:
> The 'proc *' is not used for PRU_CONTROL request, so remove it from
> pru_control() wrapper.
>
> I want to use existing in{6,}_control for tcp(4) and udp(4) sockets, so
> for inet6 case I introduced `tcp6_usrreqs' and `udp6_usrreqs'
> structures. I also want to use them for the following PRU_SOCKADDR and
> PRU_PEERADDR.
>
> Since the PRU_SOCKADDR and PRU_PEERADDR are the last ones, I want to
> move the corresponding (*pru_)() handlers and kill (*pru_usrreq)() with
> single diff. Is it ok to review?
>
> Index: sys/sys/protosw.h
> ===
> RCS file: /cvs/src/sys/sys/protosw.h,v
> retrieving revision 1.51
> diff -u -p -r1.51 protosw.h
> --- sys/sys/protosw.h 1 Sep 2022 18:21:23 - 1.51
> +++ sys/sys/protosw.h 1 Sep 2022 19:35:08 -
> @@ -59,6 +59,7 @@ struct socket;
> struct domain;
> struct proc;
> struct stat;
> +struct ifnet;
>
> struct pr_usrreqs {
> /* user request: see list below */
> @@ -77,6 +78,8 @@ struct pr_usrreqs {
> int (*pru_send)(struct socket *, struct mbuf *, struct mbuf *,
> struct mbuf *);
> int (*pru_abort)(struct socket *);
> + int (*pru_control)(struct socket *, u_long, caddr_t,
> + struct ifnet *);
> int (*pru_sense)(struct socket *, struct stat *);
> int (*pru_rcvoob)(struct socket *, struct mbuf *, int);
> int (*pru_sendoob)(struct socket *, struct mbuf *, struct mbuf
> *,
> @@ -343,12 +346,12 @@ pru_abort(struct socket *so)
> }
>
> static inline int
> -pru_control(struct socket *so, u_long cmd, caddr_t data,
> -struct ifnet *ifp, struct proc *p)
> +pru_control(struct socket *so, u_long cmd, caddr_t data, struct ifnet
> *ifp)
> {
> - return (*so->so_proto->pr_usrreqs->pru_usrreq)(so,
> - PRU_CONTROL, (struct mbuf *)cmd, (struct mbuf *)data,
> - (struct mbuf *)ifp, p);
> + if (so->so_proto->pr_usrreqs->pru_control)
> + return (*so->so_proto->pr_usrreqs->pru_control)(so,
> + cmd, data, ifp);
> + return (EOPNOTSUPP);
> }
>
> static inline int
> Index: sys/kern/sys_socket.c
> ===
> RCS file: /cvs/src/sys/kern/sys_socket.c,v
> retrieving revision 1.53
> diff -u -p -r1.53 sys_socket.c
> --- sys/kern/sys_socket.c 14 Aug 2022 01:58:28 - 1.53
> +++ sys/kern/sys_socket.c 1 Sep 2022 19:35:07 -
> @@ -137,7 +137,7 @@ soo_ioctl(struct file *fp, u_long cmd, c
> if (IOCGROUP(cmd) == 'r')
> return (EOPNOTSUPP);
> KERNEL_LOCK();
> - error = pru_control(so, cmd, data, NULL, p);
> + error = pru_control(so, cmd, data, NULL);
> KERNEL_UNLOCK();
> break;
> }
> Index: sys/kern/uipc_usrreq.c
> ===
> RCS file: /cvs/src/sys/kern/uipc_usrreq.c,v
> retrieving revision 1.182
> diff -u -p -r1.182 uipc_usrreq.c
> --- sys/kern/uipc_usrreq.c 1 Sep 2022 18:21:22 - 1.182
> +++ sys/kern/uipc_usrreq.c 1 Sep 2022 19:35:07 -
> @@ -219,8 +219,6 @@ uipc_usrreq(struct socket *so, int req,
> struct socket *so2;
> int error = 0;
>
> - if (req == PRU_CONTROL)
> - return (EOPNOTSUPP);
> if (req != PRU_SEND && control && control->m_len) {
> error = EOPNOTSUPP;
> goto release;
> Index: sys/net/if.c
> ===
> RCS file: /cvs/src/sys/net/if.c,v
> retrieving revision 1.663
> diff -u -p -r1.663 if.c
> --- sys/net/if.c13 Aug 2022 21:01:46 - 1.663
> +++ sys/net/if.c1 Sep 2022 19:35:07 -
> @@ -2360,7 +2360,7 @@ forceup:
> break;
> /* FALLTHROUGH */
> default:
> - error = pru_control(so, cmd, data, ifp, p);
> + error = pru_control(so, cmd, data, ifp);
> if (error != EOPNOTSUPP)
> break;
> switch (cmd) {
> Index: sys/net/pfkeyv2.c
> ===
> RCS file: /cvs/src/sys/net/pfkeyv2.c,v
> retrieving revision 1.249
> diff -u -p -r1.249 pfkeyv2.c
> --- sys/net/pfkeyv2.c 1 Sep 2022 18:21:23 - 1.249
> +++ sys/net/pfkeyv2.c 1 Sep 2022 19:35:07 -
> @@ -395,9 +395,6 @@ pfkeyv2_usrreq(struct socket *so, int re
> struct
Re: add sendmmsg and recvmmsg systemcalls
On Tue, Aug 30, 2022 at 11:18 AM Moritz Buhl wrote:
> the following diff only contains recvmmsg which should be the more useful
> syscall of the two.
>
Comments inline.
> --- sys/kern/syscalls.master1 Aug 2022 14:56:59 - 1.229
> +++ sys/kern/syscalls.master30 Aug 2022 15:44:29 -
> @@ -575,3 +575,6 @@
> 328OBSOL __tfork51
> 329STD NOLOCK { void sys___set_tcb(void *tcb); }
> 330STD NOLOCK { void *sys___get_tcb(void); }
> +331STD NOLOCK { int sys_recvmmsg(int s, struct mmsghdr *mmsg, \
> + unsigned int vlen, unsigned int flags, \
> + struct timespec *timeout); }
>
We believe in packing syscall numbers down, modulus having them group
nicely. So, instead of putting recvmmsg() as a new high-water mark, I
would put it at 116 ("t32_gettimeofday") and mark 117 ("t32_getrusage") as
sendmmsg(), UNIMPL if not part of this round.
(Typically, when a diff changes syscalls.master then you leave out the diff
chunks for the generated files when sending for review, because they are
100% implied and are just noise. Not a big deal)
--- sys/kern/uipc_syscalls.c14 Aug 2022 01:58:28 - 1.201
> +++ sys/kern/uipc_syscalls.c30 Aug 2022 17:03:09 -
> @@ -805,6 +805,135 @@ done:
> }
>
> int
> +sys_recvmmsg(struct proc *p, void *v, register_t *retval)
> +{
> + struct sys_recvmmsg_args /* {
> + syscallarg(int) s;
> + syscallarg(struct mmsghdr *)mmsg;
> + syscallarg(unsigned int)vlen;
> + syscallarg(unsigned int)flags;
> + syscallarg(struct timespec *) timeout;
> + } */ *uap = v;
> + struct mmsghdr mmsg;
> + struct timespec ts, now;
> + struct iovec aiov[UIO_SMALLIOV], *uiov, *iov = aiov;
> + struct file *fp;
> + struct socket *so;
> + struct timespec *timeout;
> + unsigned int vlen, dg;
> + int error = 0, flags, s;
> +
> + timeout = SCARG(uap, timeout);
> + if (timeout != NULL) {
> + error = copyin(SCARG(uap, timeout), , sizeof(ts));
> + if (error != 0)
> + return error;
>
Should have a KTRACE ktrreltimmespec() block here.
Should validate the timespec.
(Follow what sys_kevent() does)
+ getnanotime();
> + timespecadd(, , );
> + }
> +
> + s = SCARG(uap, s);
> + if ((error = getsock(p, s, )) != 0)
> + return (error);
> + so = (struct socket *)fp->f_data;
> +
> + flags = SCARG(uap, flags);
> +
> + vlen = SCARG(uap, vlen);
> + if (vlen > 1024)
> + vlen = 1024;
> +
> + for (dg = 0; dg < vlen;) {
> + error = copyin(SCARG(uap, mmsg) + dg, , sizeof(mmsg));
> + if (error != 0)
> + break;
>
Hmm. This copies in each mmsghdr structure when it gets to it. Is that
how the Linux version behaves, lazily accessing them such that an early
exit (from timeout, signal, whatever) means later values aren't read? Or
do they copy them all in, update any that are changed, then copy them out
at the end?
(Not sure it matters, but it's an interesting corner case to think
carefully about.)
...
> + mmsg.msg_hdr.msg_iov = uiov;
> + mmsg.msg_len = *retval;
> +#ifdef KTRACE
> + if (KTRPOINT(p, KTR_STRUCT)) {
> + ktrmsghdr(p, _hdr);
>
I think you should go ahead and define ktrmmsghdr() taking that full
struct, so kdump can report the msg_len value that is being returned.
> + if (iov != aiov) {
> + free(iov, M_IOV, sizeof(struct iovec) *
> + mmsg.msg_hdr.msg_iovlen);
> + iov = aiov;
> + }
>
The iov freeing, IMO,, should be done once, at the end of the loop. Just
keep growing as necessary (tracking the currently allocated size) and free
once.
kdump.c will need at least a SYS_recvmmsg line in the big table, and if you
do a ktrmmsghdr() bit in the kernel a matching decoder will be needed in
kdump.
Philip Guenther
Re: use volatile not __volatile
On Sat, Aug 27, 2022 at 6:31 PM Jonathan Gray wrote: > directly use ansi volatile keyword not __volatile from cdefs.h > Yay! ok guenther@
Re: use volatile not __volatile__
On Sat, Aug 27, 2022 at 6:41 PM Jonathan Gray wrote: > directly use ansi volatile keyword not __volatile__ builtin > Yes, please. ok guenther@
Re: static inline, not inline static
On Sun, Aug 28, 2022 at 2:11 PM Anders Andersson wrote:
> On Sun, Aug 28, 2022 at 3:15 PM Jonathan Gray wrote:
> >
> > diff --git lib/libc/locale/wctoint.h lib/libc/locale/wctoint.h
> > index ea50c5ae1b6..14c7f0c466d 100644
> > --- lib/libc/locale/wctoint.h
> > +++ lib/libc/locale/wctoint.h
> > @@ -30,7 +30,7 @@
> > */
> >
> >
> > -inline static int
> > +static inline int
> > wctoint(wchar_t wc)
> > {
> > int n;
> > [...]
>
> Why this change? As far as I can see, the standard allows for any order.
>
C99 standard stated:
"The placement of a storage-class specifier other than at the beginning
of the declaration
specifiers in a declaration is an obsolescent feature.
My recall is that it was officially removed in C11.
ok guenther@
Re: move PRU_RCVD request to (*pru_rcvd)()
Since pru_rcvd() is only invoked if the protocol has the PR_WANTRCVD flag
set, there should be no need to test whether the callback is set: a
protocol without the callback MUST NOT have PR_WANTRCVD.
(I guess this could, alternatively, go the other direction and eliminate
PR_WANTRCVD and use the presence of the callback to decide whether the
protocol needs anything to be done.)
Side note: pru_rcvd() (and the pru_rcvd implementations) should have a
return type of void.
Philip Guenther
On Mon, Aug 22, 2022 at 1:40 PM Vitaliy Makkoveev wrote:
> Another one.
>
> Since we never use `flags' arg within handlers, remove it from the
> pru_rcvd() args.
>
> Index: sys/sys/protosw.h
> ===
> RCS file: /cvs/src/sys/sys/protosw.h,v
> retrieving revision 1.43
> diff -u -p -r1.43 protosw.h
> --- sys/sys/protosw.h 22 Aug 2022 21:18:48 - 1.43
> +++ sys/sys/protosw.h 22 Aug 2022 22:27:08 -
> @@ -72,6 +72,7 @@ struct pr_usrreqs {
> int (*pru_accept)(struct socket *, struct mbuf *);
> int (*pru_disconnect)(struct socket *);
> int (*pru_shutdown)(struct socket *);
> + int (*pru_rcvd)(struct socket *);
> };
>
> struct protosw {
> @@ -314,10 +315,11 @@ pru_shutdown(struct socket *so)
> }
>
> static inline int
> -pru_rcvd(struct socket *so, int flags)
> +pru_rcvd(struct socket *so)
> {
> - return (*so->so_proto->pr_usrreqs->pru_usrreq)(so,
> - PRU_RCVD, NULL, (struct mbuf *)(long)flags, NULL, curproc);
> + if (so->so_proto->pr_usrreqs->pru_rcvd)
> + return (*so->so_proto->pr_usrreqs->pru_rcvd)(so);
> + return (EOPNOTSUPP);
> }
>
> static inline int
> Index: sys/kern/uipc_socket.c
> ===
> RCS file: /cvs/src/sys/kern/uipc_socket.c,v
> retrieving revision 1.284
> diff -u -p -r1.284 uipc_socket.c
> --- sys/kern/uipc_socket.c 21 Aug 2022 16:22:17 - 1.284
> +++ sys/kern/uipc_socket.c 22 Aug 2022 22:27:08 -
> @@ -1156,7 +1156,7 @@ dontblock:
> SBLASTRECORDCHK(>so_rcv, "soreceive 4");
> SBLASTMBUFCHK(>so_rcv, "soreceive 4");
> if (pr->pr_flags & PR_WANTRCVD && so->so_pcb)
> - pru_rcvd(so, flags);
> + pru_rcvd(so);
> }
> if (orig_resid == uio->uio_resid && orig_resid &&
> (flags & MSG_EOR) == 0 && (so->so_state & SS_CANTRCVMORE) ==
> 0) {
> @@ -1521,7 +1521,7 @@ somove(struct socket *so, int wait)
> if (m == NULL) {
> sbdroprecord(so, >so_rcv);
> if (so->so_proto->pr_flags & PR_WANTRCVD && so->so_pcb)
> - pru_rcvd(so, 0);
> + pru_rcvd(so);
> goto nextpkt;
> }
>
> @@ -1627,7 +1627,7 @@ somove(struct socket *so, int wait)
>
> /* Send window update to source peer as receive buffer has
> changed. */
> if (so->so_proto->pr_flags & PR_WANTRCVD && so->so_pcb)
> - pru_rcvd(so, 0);
> + pru_rcvd(so);
>
> /* Receive buffer did shrink by len bytes, adjust oob. */
> state = so->so_state;
> Index: sys/kern/uipc_usrreq.c
> ===
> RCS file: /cvs/src/sys/kern/uipc_usrreq.c,v
> retrieving revision 1.174
> diff -u -p -r1.174 uipc_usrreq.c
> --- sys/kern/uipc_usrreq.c 22 Aug 2022 21:18:48 - 1.174
> +++ sys/kern/uipc_usrreq.c 22 Aug 2022 22:27:08 -
> @@ -136,6 +136,7 @@ const struct pr_usrreqs uipc_usrreqs = {
> .pru_accept = uipc_accept,
> .pru_disconnect = uipc_disconnect,
> .pru_shutdown = uipc_shutdown,
> + .pru_rcvd = uipc_rcvd,
> };
>
> void
> @@ -243,32 +244,6 @@ uipc_usrreq(struct socket *so, int req,
> }
> break;
>
> - case PRU_RCVD:
> - switch (so->so_type) {
> -
> - case SOCK_DGRAM:
> - panic("uipc 1");
> - /*NOTREACHED*/
> -
> - case SOCK_STREAM:
> - case SOCK_SEQPACKET:
> - if ((so2 = unp_solock_peer(so)) == NULL)
> - break;
> - /*
> -* Adjust backpressure on sender
> -* and wakeup any waiting to write.
> -*/
>
Re: move PRU_BIND request to (*pru_bind)() handler
On Fri, Aug 19, 2022 at 12:42 PM Vitaliy Makkoveev wrote: > bluhm@ pointed, that many KASSERT()s are not welcomed, so I didn't > insert them into newly introduced handlers. Anyway except the tcp(4) > protocol, `so_pcb' cant be NULL here. But the socket lock assertion > looks reasonable. > > Some unp_*() functions could be merged with newly introduced uipc_*(), > but I want to do this after (*pru_usrreq)() split finished. > Having multiple PROTO_bind() routines that just return EOPNOTSUPP seems like overkill to me. I think I would tend to just have the pru_bind() inline do a NULL test and return EOPNOTSUPP if it is and leave the callback NULL for all those protocols, but I could also see having a single prubind_eopnotsupp() (or whatever you think is a clear name) implementation in some sys/kern/ file which could then be used by all the protocols that don't implement it. Consider how many protocols and callbacks you're going to be working through and how many stubs that would be if there's no sharing... Philip
Re: missing warning in wireguard manual page
On Mon, Jul 25, 2022 at 7:20 AM Theo de Raadt wrote: > I've been watching conversation on a mailing list, and it leads me to > wonder if we should inform the userbase better. > Too true. Certification *is* the key thing that protects users, not careful, well engineered designs. We should be giving this warning in many other places too; for example: Index: stdlib/malloc.3 === RCS file: /data/src/openbsd/src/lib/libc/stdlib/malloc.3,v retrieving revision 1.129 diff -u -p -r1.129 malloc.3 --- stdlib/malloc.3 31 Mar 2022 17:27:16 - 1.129 +++ stdlib/malloc.3 25 Jul 2022 20:00:07 - @@ -766,6 +766,11 @@ and functions appeared in .Ox 6.6 . .Sh CAVEATS +Layout randomization in +.Nm malloc +uses uncertified random number generators, +so the security properties cannot be guaranteed. +.Pp When using .Fn malloc , be wary of signed integer and
Re: echo(1): check for stdio errors
On Sun, Jul 10, 2022 at 1:08 PM Scott Cheloha
wrote:
> ok?
>
> Index: echo.c
> ===
> RCS file: /cvs/src/bin/echo/echo.c,v
> retrieving revision 1.10
> diff -u -p -r1.10 echo.c
> --- echo.c 9 Oct 2015 01:37:06 - 1.10
> +++ echo.c 10 Jul 2022 22:00:18 -
> @@ -53,12 +53,15 @@ main(int argc, char *argv[])
> nflag = 0;
>
> while (*argv) {
> - (void)fputs(*argv, stdout);
> - if (*++argv)
> - putchar(' ');
> + if (fputs(*argv, stdout) == EOF)
> + err(1, "stdout");
> + if (*++argv && putchar(' ') == EOF)
> + err(1, "stdout");
> }
> if (!nflag)
> putchar('\n');
> + if (fclose(stdout) == EOF)
> + err(1, "stdout");
>
> return 0;
> }
>
>
Three thoughts:
1) Since stdio errors are sticky, is there any real advantage to checking
each call instead of just checking the final fclose()?
2) It is claimed that POSIX requires *all* standard utilities to return
failure if any writes to stdout failed. If it's important to fix this in
'echo', can we sketch out how we might fix make fixing this in the other
utilities easier? I believe at least some chunk of the GNU utilities now
use an atexit() handler to do that; is that a reasonable idea and if not
can we come up with something better?
3) It would be no end of surprise and frustration for the external echo
utility to behave differently than the sh builtin.
Philip
Re: start unlocking kbind(2)
On Mon, 13 Jun 2022, Theo de Raadt wrote:
> Scott Cheloha wrote:
> > > Am I wrong that kbind is never called twice in the same address space?
> >
> > Isn't this exactly what happened the last time we tried this?
>
> Tried what? kbind has never been NOLOCK.
Scott's referring to rev 1.237 of kern_fork.c, where we tried to require
the first use of kbind be before __tfork(2) was called. This blew up
because there's at least two ways to legitimately call pthread_create(3)
before doing your first lazy binding:
1) build with -znow, then dlopen() something not linked with -znow after
calling pthread_create()
2) pass address of pthread_create() to another function which calls
through the pointer, no special link options required (just need to
split up the _create and *funcptr enough that the compiler
won't optimize to a direct call)
I've thought about how to possibly force a lazy resolution and haven't
thought up anything that wasn't unmaintainable, and probably
unimplementable.
So, what could work reliably?
My first thought would be to have ld.so's _dl_boot() do the equivalent of
struct __kbind kb = {
.kb_addr = _syscall_in_dl_bind;
.kb_size = 0;
};
kbind(, sizeof kb, pcookie)
...after teaching the kernel to accept such a first call to kbind and set
pr->ps_kbind_* from them. That would permit the reimposing of the "first
kbind can't be after __tfork" restriction; is it indeed enough to permit
the ps_kbind_* members without locks?
Another idea is to discard the cookie restriction and just trust the
calling address to be enough. I mean, we trust it to be sufficient
for sigreturn and that's perhaps more powerful, no? If we're fine with
that, then how about giving ld.so a PT_OPENBSD_KBIND segment with the
required address, so the kernel can just set it up at exec time?
Eventually, we could disable kbind if said header wasn't present on the ELF
interpreter.
(In a groteque abuse of the ELF program header, we *could* keep the cookie
by, for example, having the PT_OPENBSD_KBIND p_vaddr/p_memsz identify the
location of the random cookie in ld.so memory and pass the kbind syscall
address via p_paddr. You are permitted to barf.)
Philip
Re: Picky, but much more efficient arc4random_uniform!
On Tue, May 17, 2022 at 1:10 PM Steffen Nurpmeso wrote:
> Joerg Sonnenberger wrote in
> :
> |Am Fri, May 13, 2022 at 09:43:26AM -0500 schrieb Luke Small:
> |> I made a couple new versions of a new kind of arc4random_uniform-like
> ...
> |If your main use case is limiting the amount of cryptography when using
> |small bounds, there is a much simpler approach to be taken here. For
> |boundaries below 256, use arc4random_buf to extract one byte if bound is
> |a power of two, otherwise two. This gives most of the performance
> |benefit without complicating the algorithm. Extracting two bytes ensures
> |that the propability of success is > 99% and the double extracting
> |doesn't eat up the benefits.
>
> You can use (really implemented) _buf() if you need a 8-bit or
> 16-bit etc number.
>
> I find that _uniform() often makes no difference to a simple
> modulo because like the comment in _uniform() says "p > 0.5 (worst
case, usually far better", and usually RNGs sprinkle bits nicely,
>
What does that statement mean? You seem to be saying "module is uniform,
except when it isn't, which could be almost half the time for some cases,
but when it's uniform it's uniform, so why bother making it actually
correct and dependable".
I mean, what does that _mean_??? It's as if I said "my text handling
program handles all characters uniformly, except those with accents, but
that's less than 10% of the characters I type, so it handles all characters
uniformly." WTF, NO!
> 0 bytes "do not occur", so a 32-bit RNG value "is" >=0x01FF in
> most cases for "my RNG" (of 10 803/759/793 NOT; 776/805/793
> NOT for Linux getrandom(2)), which is a pretty high cut off.
> Using _uniform() just because of its name seems strange thus.
>
Where do these ideas come from, that "0 bytes 'do not occur'"?? If your
rand generator doesn't provide zero bytes at the expected frequency, you
know, 1 in 256, then you're using a garbage random number generator.
Please stop making such suggestions here because THEY ARE NOT TRUE ABOUT
OPENBSD. Do ya'll not bother to test the claims that you make?
: bleys; cat f.c
#include
#include
int
main(void)
{
uint32_t u;
long count;
count = 0;
while ((u = arc4random()) > 0x1ff)
count++;
printf("%08x\t%ld\n", u, count);
count = 0;
for (;;) {
u = arc4random();
if ((u & 0xff00) == 0 ||
(u & 0x00ff) == 0 ||
(u & 0xff00) == 0 ||
(u & 0x00ff) == 0)
break;
count++;
}
printf("%08x\t%ld\n", u, count);
return 0;
}
: bleys; cc f.c
: bleys; ./a.out
00b82e5c58
ab47880036
: bleys;
Philip Guenther
Re: Picky, but much more efficient arc4random_uniform!
On Sun, 15 May 2022, Luke Small wrote: > The current implementation is nothing more than a naive arc4random() % > upper_bound which trashes initial arc4random() calls it doesn’t like, then > transforms over a desired modulus. The whole transformation by modulus of > perfectly decent random data seems so awkward. It’s not like it is used as > some majestic artistry of RSA it seems like an ugly HACK to simply meet a > demand lacking of something better. You fail to mention correctness at all or address the fact that your version isn't while the current one is. Meanwhile, you talk about getting only just enough random data as if there's some sort of limited supply when there isn't. "My version may be wrong, but at least it doesn't look naive!" That is utterly the wrong attitude for OpenBSD code. Best wishes. Philip Guenther
Re: Picky, but much more efficient arc4random_uniform!
On Sun, 15 May 2022, Steffen Nurpmeso wrote:
> Stuart Henderson wrote in
...
> |what's the perceived problem you're wanting to solve? and does that
> |problem actually exist in the first place?
>
> The problem is that if have a low upper bound then modulo will "remove a
> lot of randomization". For example if you have a program which
> generates Lotto numbers (1..49), then using _uniform() as it is will
> generate many duplicates.
Wut. The *WHOLE POINT* of arc4random_uniform() is that it has uniform
distribution. Says right so in the manpage. If an implementation of that
API fails to do that, it's a broken implementation.
The OpenBSD implementation achieves that. NetBSD's implementation has the
same core logic. Here's my quick lotto pick demo program, doing 490
picks, so they should all be somewhere near 10:
#include
#include
#define LIMIT 49
int
main(void)
{
unsigned counts[LIMIT] = { 0 };
int i;
for (i = 0; i < 10 * LIMIT; i++)
counts[arc4random_uniform(LIMIT)]++;
for (i = 0; i < LIMIT; i++)
printf("%2d\t%7u\n", i+1, counts[i]);
return 0;
}
And a sample run:
: bleys; ./a.out
1 100100
2 100639
399965
499729
599641
699650
7 100299
8 100164
999791
10 100101
11 100657
12 100042
1399661
1499927
1599426
1699491
1799646
18 100133
19 100013
2099942
2199873
2299924
2399567
24 100152
25 100688
26 100011
27 100481
2899980
29 100406
3099726
3199808
3299929
33 100050
3499983
35 100048
3699771
3799906
38 100215
39 100261
40 100426
4199847
4299533
43 100368
4499695
45 100041
46 100465
4799875
48 100034
4999920
: bleys;
Looks pretty good to me, with repeated runs showing the values bouncing around.
...
> I was a bit interested when i saw Luke's message, but i am no
> mathematician and, like i said, i in fact never needed the
> functionality. And the question i would have is how small
> upper_bound has to be for the modulo problem to really kick in.
If the implementation isn't broken, it's not a problem, period.
> And even if, whether it matters. For example, if you have
> a bitset of 1024 "in-flight things", and the small upper bound
> hits a used slot, you could simply search forward until you find
> a free one. I mean, with 1024 possible values the number of
> possibilities is anyhow restricted.
Well hey, let's give Luke's a try and see how uniform it is:
#include
#include
#define LIMIT 49
int
main(void)
{
unsigned counts[LIMIT] = { 0 };
unsigned counts2[LIMIT] = { 0 };
int i;
for (i = 0; i < 10 * LIMIT; i++) {
counts[arc4random_uniform(LIMIT)]++;
counts2[arc4random_uniform_fast_simple(LIMIT)]++;
}
for (i = 0; i < LIMIT; i++)
printf("%2d\t%7u\t%7u\n", i+1, counts[i], counts2[i]);
return 0;
}
: bleys; ./a.out
1 100188 76502
299983 76602
3 100521 76522
4 100416 76682
5 100171 76387
6 100163 76759
7 100024 76336
8 19 76703
999769 76237
1099892 76532
11 100197 76730
12 100483 76398
1399769 76310
14 100075 76474
1599781 76599
1699846 76439
1799814 76430
18 100313 76648
19 100259 76813
2099885 77068
21 100302 76546
228 76698
2399491 76678
24 100340 76324
2599763 115263
2699872 153008
27 100022 152979
2899481 153793
29 100018 210714
3099617 229286
31 100167 297003
32 100270 449664
33 100468 76790
3499115 76452
3599921 76392
3699862 76140
37 100485 76607
38 100029 75885
3999577 76498
4099479 76727
41 100139 76746
42 100883 76698
43 100102 76474
4499801 76592
45 100117 76124
4699678 76417
4799770 76639
4899524 77034
49 100151 76658
: bleys;
Wow, that last column is *bad*. Repeated runs show 25-32 are consistently
high, 32 being *outrageously* off, and the others are low.
Luke's implementation does not correctly implement the API. Doesn't
matter if it's a million times faster when it doesn't deliver the goal.
Philip Guenther
Re: stop using mquery(2) inside realloc(3)
On Sat, 14 May 2022, Philip Guenther wrote: > On Sat, 14 May 2022, Theo de Raadt wrote: > > I worry a little about having libc use an undocumented mmap(2) flag. > > About as much as using mquery, which is non-standard. > > > > Maybe __MAP_NOREPLACE should get documentation? __MAP_NOFAULT is in the > > same situation. The behaviour of these flags should be documented > > (set in stone), which may also discourage accidental behaviour changes > > by kernel developers in the future? > > To throw some spaghetti at the wall... Fix the grammar of the __MAP_NOFAULT description and mention signals there. Index: sys/mmap.2 === RCS file: /data/src/openbsd/src/lib/libc/sys/mmap.2,v retrieving revision 1.68 diff -u -p -r1.68 mmap.2 --- sys/mmap.2 31 Mar 2022 17:27:16 - 1.68 +++ sys/mmap.2 14 May 2022 19:31:28 - @@ -58,8 +58,19 @@ The argument describes the address where the system should place the mapping. If the .Dv MAP_FIXED -flag is specified, the allocation will happen at the specified address, +flag is specified and the +.Dv __MAP_NOREPLACE +flag is not specified, +the allocation will happen at the specified address, replacing any previously established mappings in its range. +If both the +.Dv MAP_FIXED +and +.Dv __MAP_NOREPLACE +flags are specified, +the allocation will happen at the specified address, +failing with no effect if any previously established mappings are +in its range. Otherwise, the mapping will be placed at the available spot at .Fa addr ; failing that it will be placed "close by". @@ -153,6 +164,18 @@ mappings) must be multiples of the page size. Existing mappings in the address range will be replaced. Use of this option is discouraged. +.It Dv __MAP_NOFAULT +Indicate that access to pages that are not backed by the mapped +file or device will result in zero-filled anonymous pages being +provided instead of a signal being delivered. +This flag must not be used with an anonymous mapping. +.It Dv __MAP_NOREPLACE +Indicates that a +.Dv MAP_FIXED +mapping should fail if it would require replacing any existing +mappings. +This flag must be used in combination with +.Dv MAP_FIXED . .It Dv MAP_STACK Indicate that the mapping is used as a stack. This flag must be used in combination with @@ -278,6 +301,12 @@ device does not support memory mapping. The allocation .Fa len was 0. +.It Bq Er EINVAL +.Dv __MAP_NOFAULT +was specified for an anonymous mapping or +.Dv __MAP_NOREPLACE +was specified without +.Dv MAP_FIXED . .It Bq Er ENOMEM .Dv MAP_FIXED was specified and the @@ -323,6 +352,14 @@ A fully functional system call first appeared in SunOS 4.0 and has been available since .Bx 4.3 Net/2 . +The +.Dv __MAP_NOFAULT +flag appeared in +.Ox 5.7 . +The +.Dv __MAP_NOREPLACE +flag appeared in +.Ox 5.3 . .Sh CAVEATS .St -p1003.1-2008 specifies that references to pages beyond the end of a mapped object
Re: stop using mquery(2) inside realloc(3)
On Sat, 14 May 2022, Theo de Raadt wrote: > I worry a little about having libc use an undocumented mmap(2) flag. > About as much as using mquery, which is non-standard. > > Maybe __MAP_NOREPLACE should get documentation? __MAP_NOFAULT is in the > same situation. The behaviour of these flags should be documented > (set in stone), which may also discourage accidental behaviour changes > by kernel developers in the future? To throw some spaghetti at the wall... Index: sys/mmap.2 === RCS file: /data/src/openbsd/src/lib/libc/sys/mmap.2,v retrieving revision 1.68 diff -u -p -r1.68 mmap.2 --- sys/mmap.2 31 Mar 2022 17:27:16 - 1.68 +++ sys/mmap.2 14 May 2022 19:06:00 - @@ -58,8 +58,19 @@ The argument describes the address where the system should place the mapping. If the .Dv MAP_FIXED -flag is specified, the allocation will happen at the specified address, +flag is specified and the +.Dv __MAP_NOREPLACE +flag is not specified, +the allocation will happen at the specified address, replacing any previously established mappings in its range. +If both the +.Dv MAP_FIXED +and +.Dv __MAP_NOREPLACE +flags are specified, +the allocation will happen at the specified address, +failing with no effect if any previously established mappings are +in its range. Otherwise, the mapping will be placed at the available spot at .Fa addr ; failing that it will be placed "close by". @@ -153,6 +164,17 @@ mappings) must be multiples of the page size. Existing mappings in the address range will be replaced. Use of this option is discouraged. +.It Dv __MAP_NOFAULT +Indicate that access to pages that are not backed by the mapped +file or device will be replaced by zero-filled anonymous pages. +This flag must not be used with an anonymous mapping. +.It Dv __MAP_NOREPLACE +Indicates that a +.Dv MAP_FIXED +mapping should fail if it would require replacing any existing +mappings. +This flag must be used in combination with +.Dv MAP_FIXED . .It Dv MAP_STACK Indicate that the mapping is used as a stack. This flag must be used in combination with @@ -278,6 +300,12 @@ device does not support memory mapping. The allocation .Fa len was 0. +.It Bq Er EINVAL +.Dv __MAP_NOFAULT +was specified for an anonymous mapping or +.Dv __MAP_NOREPLACE +was specified without +.Dv MAP_FIXED . .It Bq Er ENOMEM .Dv MAP_FIXED was specified and the @@ -323,6 +351,14 @@ A fully functional system call first appeared in SunOS 4.0 and has been available since .Bx 4.3 Net/2 . +The +.Dv __MAP_NOFAULT +flag appeared in +.Ox 5.7 . +The +.Dv __MAP_NOREPLACE +flag appeared in +.Ox 5.3 . .Sh CAVEATS .St -p1003.1-2008 specifies that references to pages beyond the end of a mapped object
stop using mquery(2) inside realloc(3)
If you try to grow a 'large' (at least half a page) allocation with
realloc(3), it'll try to extend the memory mapping for it and if that
works it won't need to move it.
Currently, it does that by using mquery(2) to see if that area is
available and if so then trying to mmap it, munmaping it if mmap(2) didn't
return the desired address (perhaps due to a race with another thread).
Instead of doing mquery/mmap/munmap, this path can use the MAP_FIXED and
__MAP_NOREPLACE flags to directly request the specific address but not
change anything if it's not available.
(We still use mquery in ld.so on i386 as a performance optimization, but
that use case differs in needing to find a base address for *multiple*
mappings, where unrolling partial hits grows to be more expensive than
probing with mquery and only trying to do all the mapping on a successful
set of probes: recall that on x86 munmap() is more expensive than mmap()
due to TLB shootdowns. This case in realloc() only has a single mapping to
probe/extend so it avoids those costs. Indeed, this diff eliminates the
current "munmap on failed attempt" path/cost.)
The regress/lib/libc/malloc tests continue to pass on my amd64 box, with
ktrace confirming the change in calls.
oks?
Philip
Index: stdlib/malloc.c
===
RCS file: /data/src/openbsd/src/lib/libc/stdlib/malloc.c,v
retrieving revision 1.273
diff -u -p -r1.273 malloc.c
--- stdlib/malloc.c 26 Feb 2022 16:14:42 - 1.273
+++ stdlib/malloc.c 14 May 2022 02:35:04 -
@@ -100,9 +100,6 @@
#define MMAPA(a,sz,f) mmap((a), (sz), PROT_READ | PROT_WRITE, \
MAP_ANON | MAP_PRIVATE | (f), -1, 0)
-#define MQUERY(a,sz,f) mquery((a), (sz), PROT_READ | PROT_WRITE, \
-MAP_ANON | MAP_PRIVATE | MAP_FIXED | (f), -1, 0)
-
struct region_info {
void *p;/* page; low bits used to mark chunks */
uintptr_t size; /* size for pages, or chunk_info pointer */
@@ -1687,11 +1684,7 @@ orealloc(struct dir_info **argpool, void
size_t needed = rnewsz - roldsz;
STATS_INC(pool->cheap_realloc_tries);
- q = MQUERY(hint, needed, pool->mmap_flag);
- if (q == hint)
- q = MMAPA(hint, needed,
pool->mmap_flag);
- else
- q = MAP_FAILED;
+ q = MMAPA(hint, needed, MAP_FIXED |
__MAP_NOREPLACE | pool->mmap_flag);
if (q == hint) {
STATS_ADD(pool->malloc_used, needed);
if (pool->malloc_junk == 2)
@@ -1709,9 +1702,6 @@ orealloc(struct dir_info **argpool, void
STATS_INC(pool->cheap_reallocs);
ret = p;
goto done;
- } else if (q != MAP_FAILED) {
- if (munmap(q, needed))
- wrterror(pool, "munmap %p", q);
}
}
} else if (rnewsz < roldsz) {
eliminate use of fseek(3) inside libc
fseek(3) is implemented by just calling fseeko(3), the POSIX addition
version that takes an off_t instead of a long:
int
fseek(FILE *fp, long offset, int whence)
{
return (fseeko(fp, offset, whence));
}
...so there's no reason for libc to be using fseek() internally, as that
just loses range on ILP32 platforms and adds that (tail)call overhead.
Diff below converts the last three such uses** and then updates the symbol
naming for fseek to block attempts to reference it in the future so uses
don't accidentally crawl back in.
ok?
Philip
** theoretically fixing the handling of xdr_setpos() to a position
beyond 2GB on an xdrstdio, I guess
Index: gen/getcap.c
===
RCS file: /data/src/openbsd/src/lib/libc/gen/getcap.c,v
retrieving revision 1.35
diff -u -p -r1.35 getcap.c
--- gen/getcap.c3 Jul 2019 03:24:04 - 1.35
+++ gen/getcap.c23 Dec 2019 23:21:11 -
@@ -244,7 +244,7 @@ getent(char **cap, u_int *len, char **db
* Open database if not already open.
*/
if (fp != NULL) {
- (void)fseek(fp, 0L, SEEK_SET);
+ fseeko(fp, 0, SEEK_SET);
myfd = 0;
opened++;
} else {
Index: rpc/xdr_stdio.c
===
RCS file: /data/src/openbsd/src/lib/libc/rpc/xdr_stdio.c,v
retrieving revision 1.16
diff -u -p -r1.16 xdr_stdio.c
--- rpc/xdr_stdio.c 14 Feb 2022 03:38:59 - 1.16
+++ rpc/xdr_stdio.c 14 May 2022 01:40:56 -
@@ -144,7 +144,7 @@ static bool_t
xdrstdio_setpos(XDR *xdrs, u_int pos)
{
- return ((fseek((FILE *)xdrs->x_private, (long)pos, SEEK_SET) == -1) ?
+ return ((fseeko((FILE *)xdrs->x_private, pos, SEEK_SET) == -1) ?
FALSE : TRUE);
}
Index: stdio/rewind.c
===
RCS file: /data/src/openbsd/src/lib/libc/stdio/rewind.c,v
retrieving revision 1.6
diff -u -p -r1.6 rewind.c
--- stdio/rewind.c 31 Aug 2015 02:53:57 - 1.6
+++ stdio/rewind.c 23 Dec 2019 23:30:22 -
@@ -37,7 +37,7 @@
void
rewind(FILE *fp)
{
- (void) fseek(fp, 0L, SEEK_SET);
+ fseeko(fp, 0, SEEK_SET);
clearerr(fp);
errno = 0; /* not required, but seems reasonable */
}
Index: hidden/stdio.h
===
RCS file: /data/src/openbsd/src/lib/libc/hidden/stdio.h,v
retrieving revision 1.7
diff -u -p -r1.7 stdio.h
--- hidden/stdio.h 6 Sep 2016 19:56:36 - 1.7
+++ hidden/stdio.h 14 May 2022 02:01:51 -
@@ -65,7 +65,7 @@ PROTO_NORMAL(fputs);
PROTO_NORMAL(fread);
PROTO_NORMAL(freopen);
PROTO_NORMAL(fscanf);
-PROTO_NORMAL(fseek);
+PROTO_STD_DEPRECATED(fseek);
PROTO_NORMAL(fseeko);
PROTO_NORMAL(fsetpos);
PROTO_NORMAL(ftell);
Index: stdio/fseek.c
===
RCS file: /data/src/openbsd/src/lib/libc/stdio/fseek.c,v
retrieving revision 1.13
diff -u -p -r1.13 fseek.c
--- stdio/fseek.c 28 Jun 2019 13:32:42 - 1.13
+++ stdio/fseek.c 14 May 2022 02:00:03 -
@@ -250,4 +250,3 @@ fseek(FILE *fp, long offset, int whence)
{
return (fseeko(fp, offset, whence));
}
-DEF_STRONG(fseek);
Re: VMM avoid duplication and reduce atack surface with octboot(4)
On Tue, Mar 22, 2022 at 6:04 PM Alexis wrote: > > english is not my native language my point is octboot good be used to > make openbsd the bootloader on vmm therefore no need to maintaine other > upstream stacks, therefore avoiding duplication and reducing atack surface > like it was done on that project explained in that CCC talk. The responses you're getting are driven by the fact that you appear to ascribe to octboot(4/octeon) the functionality to replace an x86 BIOS. octboot(4/octeon) is a driver (chunk of code) in the MIPS64 version of the OpenBSD kernel which lets an *already running* kernel replace itself with another kernel. An x86 BIOS is the code which bootstraps an x86 with particular memory and I/O setups, providing early services for access to that I/O and memory such that other stuff, like OpenBSD boot loaders, can themselves be invoked. These are completely different functions and neither can replace the other. I'm not sure what information led you to understand these to be similar functions, but boot sequences are among the most...specific pieces of system knowledge and while some people certainly start their approach from that angle, I suspect most people dig into that only after time spent on more portable (and frankly, usable) levels. Not knowing what your interests and energy are in, I can give no specific recommendation. Philip Guenther
Re: Provide memory barriers in refcnt_rele() and refcnt_finalize()
On Mon, Mar 14, 2022 at 12:47 AM Visa Hankala wrote:
> On Sun, Mar 13, 2022 at 06:26:19PM -0700, Philip Guenther wrote:
> > On Sun, Mar 13, 2022 at 10:27 AM Visa Hankala wrote:
> >
> > > On Sun, Mar 13, 2022 at 04:29:44PM +0100, Mark Kettenis wrote:
> > >
> > ...
> >
> > > > Under what circumstances does memory ordering matter for these
> > > > interfaces?
> > >
> > > Consider the following scenario:
> > >
> > > struct widget {
> > > struct refcnt w_refcnt;
> > > /* more fields spanning many cache lines */
> > > ...
> > > int w_var;
> > > };
> > >
> > > First, CPU 1 executes:
> > >
> > > w->w_var = 1;
> > > refcnt_rele(>w_refcnt); /* remains above zero */
> > >
> >
> > Having incremented the refcnt previous does not give this thread
> exclusive
> > access to 'w', so if it's writing to w->w_var then it must either
> > a) have some sort of write lock taken, which it will release after this
> and
> > which will contain the necessary member, OR
> > b) have the only access patch to this structure (i.e, it's not yet
> > 'published' into structures which can be seen by other threads), in which
> > case the operations which do that 'publishing' of the access to 'w'
> (adding
> > it to a global list, etc) must include the necessary membar.
>
> Lets change the sequence to this:
>
> local_var = atomic_load_int(>w_var);
> refcnt_rele(>w_refcnt);
>
> Without the release barrier, is the load guaranteed to happen before
> the reference count is decremented?
>
That's completely uncomparable to what you described before for "CPU 1"
because there's no write to anything but the refcnt.
If no one writes to the object that is ref counted, then there are no
visibility problems, no?
> Next, CPU 2 executes:
> > >
> > > if (refcnt_rele(>w_refcnt)) /* refcnt drops to zero */
> > > free(w);
> > >
> >
> > How did CPU 2 get what is now exclusive access to 'w' without any
> membars?
> > If that's possible then it was just accessing 'w' and possibly not seeing
> > the update to w->w_var even _before_ the refcnt_rele(), so putting a
> membar
> > in refcnt_rele() hides the incorrect code by suppressing the later crash!
> >
> > If these membars appear to help then the code is and remains broken.
> This
> > change should not be done.
>
> It is not uncommon to see something like below:
>
> Access object with read intent:
>
> mtx_enter(_lock);
> w = lookup_from_list();
> if (w != NULL)
> refcnt_take(>w_refcnt);
> mtx_leave(_lock);
> if (w == NULL)
> return;
> ...
>
No writes to *w described *OR LEGAL*.
> if (refcnt_rele(>w_refcnt))
> free(w);
Delete object:
>
> mtx_enter(_lock);
> w = lookup_from_list();
> if (w != NULL)
> remove_from_list(w);
> mtx_leave(_lock);
>
This does the 'release' on the change to w's list link(s).
> /* Release list's reference. */
> if (w != NULL && refcnt_rele(>w_refcnt))
> free(w);
>
> Above, any refcnt_rele() can release the final reference.
>
> If there is no acquire barrier after the refcnt 1->0 transition, what
> is known about the CPU's local view of the object after refcnt_rele()?
>
Okay, if refcnt is used with objects that have embedded list links, or
could be, then refcnt_rele needs acquire semantics on the 1->0 transition.
I can agree with your justification for that.
Do you have a similar example for giving it release semantics as your diff
proposed? What's the otherwise-unprotected-by-synchronization-primitive
that has to be protected?
> The decrement operation in the API of Linux and
> the API of FreeBSD provide release and acquire
> barriers. Are they wrong?
>
I'm not sure what argument you're making here.
So far, I've understood this to be "here's why this API needs to provide
these semantics, as justified by the logic of acquire/release semantics and
examples". Per above, I think you've justified giving it acquire
semantics, but don't understand the justification for also giving it
release semantics.
Now you're giving a completely different justification: "they did so".
That's a justification under any of (at least) three lines:
a) they're incapable of making mistakes
b) you've read the docs/explanation behind the
Re: Provide memory barriers in refcnt_rele() and refcnt_finalize()
On Sun, Mar 13, 2022 at 10:27 AM Visa Hankala wrote:
> On Sun, Mar 13, 2022 at 04:29:44PM +0100, Mark Kettenis wrote:
>
...
> > Under what circumstances does memory ordering matter for these
> > interfaces?
>
> Consider the following scenario:
>
> struct widget {
> struct refcnt w_refcnt;
> /* more fields spanning many cache lines */
> ...
> int w_var;
> };
>
> First, CPU 1 executes:
>
> w->w_var = 1;
> refcnt_rele(>w_refcnt); /* remains above zero */
>
Having incremented the refcnt previous does not give this thread exclusive
access to 'w', so if it's writing to w->w_var then it must either
a) have some sort of write lock taken, which it will release after this and
which will contain the necessary member, OR
b) have the only access patch to this structure (i.e, it's not yet
'published' into structures which can be seen by other threads), in which
case the operations which do that 'publishing' of the access to 'w' (adding
it to a global list, etc) must include the necessary membar.
Next, CPU 2 executes:
>
> if (refcnt_rele(>w_refcnt)) /* refcnt drops to zero */
> free(w);
>
How did CPU 2 get what is now exclusive access to 'w' without any membars?
If that's possible then it was just accessing 'w' and possibly not seeing
the update to w->w_var even _before_ the refcnt_rele(), so putting a membar
in refcnt_rele() hides the incorrect code by suppressing the later crash!
If these membars appear to help then the code is and remains broken. This
change should not be done.
Philip Guenther
Re: ifconfig(8): always print the mtu, don't hide it on "bridges"
On Mon, Feb 21, 2022 at 9:47 PM David Gwynne wrote: > this lets ifconfig show the MTU on interfaces like nvgre, vxlan, etc. > they currently don't show it because they also implement a bridge ioctl, > so ifconfig thinks they're a bridge. > > why ifconfig hides the mtu on bridges looks to be a hold over from when > brconfig was merged into ifconfig. if we dont want bridge(4) to report > an mtu, then i can make bridge(4) itself hide the mtu or stop setting > the mtu. > > found by jason tubnor. > > ok? Ah, this test explains one of the three times ioctl(SIOCBRDGRTS) is used on each interface. This will drop it to twice per interface ;) ok guenther@
Re: Power-up cc --print-file-name for .so names
On Sun, Feb 13, 2022 at 11:29 PM Philip Guenther wrote: > On Sun, Feb 13, 2022 at 11:18 PM Mark Kettenis > wrote: > >> > From: Greg Steuck >> > Date: Sun, 13 Feb 2022 22:37:13 -0800 >> > >> > To give a sense of the kind of change required to get the feature I >> > want, see the patch at the end. The change in DriverUtils.cpp is just to >> > show that the same function is hiding in there. >> > >> > If this looks like a good direction, I can cleanup the code and maybe it >> > could be shared, though I'm not sure where this would belong. Maybe a >> > new tiny library of such wrappers to maintain in our tree? >> > >> > Thanks >> > Greg >> > >> > P.S. Naturally, once I install this patched cc, ghci is suddenly very >> > happy. >> >> Something like this was rejected upstream. >> >> The solution would be to add symlinks like all the other OSes do. But >> Theo doesn't like that. >> > > My recall was it was less that "Theo didn't like it" than "no one had > actually worked out how the symlink update process would work across major > and minor version bumps in base builds and what else needed to be adjusted > to keep things from exploding during that", because if there's one thing > Theo doesn't like it's "oh, something committed months ago and now embedded > in our ecosystem means that our ABI change process NO LONGER WORKS". > > So, IMHO, what is needed is for someone to camp out and watch for a diff > floated that will require a major bump *and that will affect stuff run > during the base build!* and then > 1) apply the symlink changes that *seem* fine, and build some dependent > ports, > 2) apply the major bump diff > 3) build base > 4) verify whether anything exploded in the base build and if so, what/why > 5) so, do those ports still work, or do they explode until an update to > them is pushed? > Those of long memory will recall a hackathon where dependencies on libc were put in place, the libm vs libc deps were changed as functions were moved from libm to libc, and base builds completely broke. My recall is that the diffs had to basically be unrolled to restore base builds to operation, and then the libm to libc stuff was redone, but I may be repressing memories. Philip
Re: Power-up cc --print-file-name for .so names
On Sun, Feb 13, 2022 at 11:18 PM Mark Kettenis wrote: > > From: Greg Steuck > > Date: Sun, 13 Feb 2022 22:37:13 -0800 > > > > To give a sense of the kind of change required to get the feature I > > want, see the patch at the end. The change in DriverUtils.cpp is just to > > show that the same function is hiding in there. > > > > If this looks like a good direction, I can cleanup the code and maybe it > > could be shared, though I'm not sure where this would belong. Maybe a > > new tiny library of such wrappers to maintain in our tree? > > > > Thanks > > Greg > > > > P.S. Naturally, once I install this patched cc, ghci is suddenly very > > happy. > > Something like this was rejected upstream. > > The solution would be to add symlinks like all the other OSes do. But > Theo doesn't like that. > My recall was it was less that "Theo didn't like it" than "no one had actually worked out how the symlink update process would work across major and minor version bumps in base builds and what else needed to be adjusted to keep things from exploding during that", because if there's one thing Theo doesn't like it's "oh, something committed months ago and now embedded in our ecosystem means that our ABI change process NO LONGER WORKS". So, IMHO, what is needed is for someone to camp out and watch for a diff floated that will require a major bump *and that will affect stuff run during the base build!* and then 1) apply the symlink changes that *seem* fine, and build some dependent ports, 2) apply the major bump diff 3) build base 4) verify whether anything exploded in the base build and if so, what/why 5) so, do those ports still work, or do they explode until an update to them is pushed? Philip
Re: [PATCH v2 3/3] script(1): fix exit status wording, use 125 for general failure
On Fri, Jan 28, 2022 at 5:28 AM наб wrote: > This is a base-line attempt at separating errors from the child from the > ones from script itself ‒ 125 is the general-purpose code in POSIX > utilities that exec() (with 126 being ENOEXEC and 127 ‒ ENOENT) > I just checked the draft of the next revision of the POSIX spec and can find no reference to 125 being either recommended or required as the status for general exec failures. For example, the spec for xargs includes this: EXIT STATUS The following exit values shall be returned: 0All invocations of utility returned exit status zero. 1-125 A command line meeting the specified requirements could not be assembled, one or more of the invocations of utility returned a non-zero exit status, or some other error occurred. 126The utility specified by utility was found but could not be invoked. 127The utility specified by utility could not be found. I'm confident that this isn't a change from previous versions. Where is this proposed use of 125 documented? Philip Guenther
Re: touch(1): don't leak descriptor if futimens(2) fails
On Fri, Jan 28, 2022 at 7:37 AM Scott Cheloha wrote: > On Fri, Jan 28, 2022 at 07:28:40AM -0700, Todd C. Miller wrote: > > On Thu, 27 Jan 2022 20:02:18 -0800, Philip Guenther wrote: > > > > > > I think futimens(2) and close(2) failures are exotic enough to > warrant > > > > printing the system call name. > > > > > > I don't understand this. Can you give an example of an error message > that > > > touch currently might emit where knowing that the failed call was > > > futimens() or close() would affect the analysis of how to deal with > it? I > > > mean, it looks like the only errors that futimens() could really > return are > > > EROFS, EIO, and EPERM (implies a race by different users to create the > > > file), and close() could only return EIO. For any of those errors, > you're > > > going to handle them the same whether they're from open, futimens, or > > > close, no? > > > > I agree. The actual syscall in this case is pretty much irrelevant. > > The mostly likely failure is due to an I/O error of some kind. > > Alright, you have both convinced me. > > We'll go with this patch? > Sure (but I think the rval assignment and warn() call should be in a consistent order). Philip Guenther
Re: touch(1): don't leak descriptor if futimens(2) fails
On Thu, Jan 27, 2022 at 4:03 PM Scott Cheloha wrote: > If futimens(2) fails here then close(2) is not called and we leak the > descriptor. > Good catch. I think futimens(2) and close(2) failures are exotic enough to warrant > printing the system call name. > I don't understand this. Can you give an example of an error message that touch currently might emit where knowing that the failed call was futimens() or close() would affect the analysis of how to deal with it? I mean, it looks like the only errors that futimens() could really return are EROFS, EIO, and EPERM (implies a race by different users to create the file), and close() could only return EIO. For any of those errors, you're going to handle them the same whether they're from open, futimens, or close, no? Philip Guenther
Re: Properly check if ACPI devices are enabled
On Mon, Jan 24, 2022 at 11:41 AM Mark Kettenis wrote: > > Date: Mon, 24 Jan 2022 20:19:46 +0100 > > From: Anton Lindqvist > > > > On Mon, Jan 24, 2022 at 05:31:49PM +0100, Mark Kettenis wrote: > > > Currently we attach ACPI devices that are present in a machine. > > > However, in some cases ACPI devices can be present, but not enabled. > > > Attaching a device driver to devices that are not enabled is not a > > > good idea since reading and writing from/to its registers will fail > > > and the driver will malfunction in interesting ways. Such as a com(4) > > > serial port that is misdetected and hangs the kernel when it is > > > actually opened. > > > > > > The diff below makes sure we only enable devices that are actually > > > enabled. This may cause some devices to disappear in OpenBSD. > > > However those devices should have been unusable anyway, so that isn't > > > an issue. > > > > > > ok? > > > > According to the ACPI specification[1]: > > > > > A device can only decode its hardware resources if both bits 0 and 1 > are > > > set. If the device is not present (bit [0] cleared) or not enabled (bit > > > [1] cleared), then the device must not decode its resources. > > Just before that it says: > > If bit [0] is cleared, then bit 1 must also be cleared (in other > words, a device that is not present cannot be enabled). > > > Should we therefore check for presence of both STA_PRESENT and > > STA_ENABLED? > > So according to the ACPI specification we don't need to do that. > Should we do it just to be safe? > Unless you're taking money bets about this being the one thing in the ACPI spec that some vendor won't screw up, doing both seems "can't be worse; can be better". Philip
Re: usr.bin/mg: fix -Wunused-but-set-variable warnings
On Sun, Jan 16, 2022 at 8:10 AM Christian Weisgerber wrote: > usr.bin/mg: fix -Wunused-but-set-variable warnings > > * strtonum() is only called to verify that a string is numerical, the > return value is unused. > * inlist is no longer used after the code was refactored. > > OK? > ok guenther@
Re: usr.sbin/dvmrpctl: fix -Wunused-but-set-variable warning
On Sun, Jan 16, 2022 at 12:51 PM Christian Weisgerber wrote: > usr.sbin/dvmrpctl: fix -Wunused-but-set-variable warning > > This looks like /* not yet implemented */, but the companion functions > show_rib_detail_msg() and show_mfc_detail_msg() are equally empty. > ok guenther@
Re: usr.sbin/eigrpd: fix -Wunused-but-set-variable warning
On Sun, Jan 16, 2022 at 1:51 PM Christian Weisgerber wrote: > usr.sbin/eigrpd: fix -Wunused-but-set-variable warning > ok guenther@
Re: usr.sbin/ospf6ctl: fix -Wunused-but-set-variable warning
On Mon, Jan 17, 2022 at 6:36 AM Christian Weisgerber wrote: > usr.sbin/ospf6ctl: fix -Wunused-but-set-variable warning > > Maybe this is uncompleted code, but I think we can remove it for > the time being. > > M usr.sbin/ospf6ctl/ospf6ctl.c > Agreed. ok guenther@
Re: dt: make vmm tracepoints amd64 only
On Mon, Jan 17, 2022 at 5:02 AM Klemens Nanni wrote: > These don't hurt on !VMM architectures but I was still surprised to see > them on e.g. sparc64: > > # arch -s ; btrace -l | grep vmm > sparc64 > tracepoint:vmm:guest_enter > tracepoint:vmm:guest_exit > > Like some network drivers, we could use __amd64__ to limit those to > amd64 and save a few bits in all other kernels. > > Is this approach feasible or should we just ignore such one-offs? > If there's a non-trivial number of trace points in drivers, then it would suggest that such tracepoints should be added/registered/pick-a-term by the driver's attach routine. Philip
Re: sbin/isakmpd: fix -Wunused-but-set-variable warnings
On Sat, Jan 15, 2022 at 12:36 PM Christian Weisgerber wrote: > sbin/isakmpd: fix -Wunused-but-set-variable warnings > > The one in pf_key_v2.c could use an extra set of eyes, but I don't > think there are any side effects. > The involved variables in pf_key_v2.c were added in 2000 as part of some sort of sync-with-upstream, weren't used then, and never used after. Deleting them seems perfectly fine to me. ok guenther@ on the entire diff
Re: new: lang/polyml
On Tue, Jan 11, 2022 at 4:09 PM Daniel Dickman wrote: > On Mon, Jan 10, 2022 at 8:12 PM Leo Larnack wrote: > > > > i386 > > > > > > with this diff I was able to install includes, rebuild ld.so and > ctfconv. I've not managed to build a release yet. ... Umm, with what diff? There was no diff in nor attached to that message. :-/ (That was a lot of lines of output. I don't know about ports@, but my expectation would be there would be *zero* reviewers of anything before, say, the last 50 lines of output before the switch to actual compilation. Standard "make lots of noise so when a failure occurs we can see the leadup, but we'll ignore it otherwise" style of output, like a base build. You read the lead up to the warnings and errors only. ) Philip Guenther
Re: new: lang/polyml
Yeah, it makes sense to move our base C environment to match the values
seen in the output of 'readelf' and in the broader programming environment.
Philip
On Mon, Jan 10, 2022 at 3:34 PM Mark Kettenis
wrote:
> > Date: Sun, 09 Jan 2022 23:54:23 +0100
> > From: Leo Larnack
> >
> > Daniel Dickman wrote:
> >
> > > elfexport.cpp:352:56: error: use of undeclared identifier 'R_386_32'
> > > reloc.r_info = ELFXX_R_INFO(symbolNum, isFuncPtr ?
> > > HOST_DIRECT_FPTR_RELOC : HOST_DIRECT_DATA_RELOC);
> > >^
> > > elfexport.cpp:142:33: note: expanded from macro
> 'HOST_DIRECT_FPTR_RELOC'
> > > # define HOST_DIRECT_FPTR_RELOC R_386_32
> >
> > It looks like Poly/ML expects R_386_32 to be either in of
> > (it's located in on Void Linux, for
> > example). After a bit of searching, what I understand is that
> > OpenBSD/i386's (and other platforms') contains
> > the right values, but doesn't follow the same naming convention
> > (RELOC_32), while other platforms do follow that convention.
> >
> > Hoping that I'm not doing something dumb, here is a diff that
> > standardizes the constants' names for these platforms. The new names
> > come from src/gnu/usr.bin/binutils-2.17/include/elf.
>
> Yes, I think this is a diff we want. But it is probably best to
> review and test this on a per-architecture basis.
>
> Can you feed them to us one-by-one?
>
>
> > Index: libexec/ld.so/hppa/rtld_machine.c
> > ===
> > RCS file: /cvs/src/libexec/ld.so/hppa/rtld_machine.c,v
> > retrieving revision 1.43
> > diff -u -p -r1.43 rtld_machine.c
> > --- libexec/ld.so/hppa/rtld_machine.c 8 Jan 2022 06:49:41 -
> 1.43
> > +++ libexec/ld.so/hppa/rtld_machine.c 9 Jan 2022 22:03:27 -
> > @@ -164,7 +164,7 @@ _dl_md_reloc(elf_object_t *object, int r
> > int type;
> >
> > type = ELF_R_TYPE(rela->r_info);
> > - if (type == RELOC_NONE)
> > + if (type == R_TYPE(NONE))
> > continue;
> >
> > sym = object->dyn.symtab + ELF_R_SYM(rela->r_info);
> > @@ -191,7 +191,7 @@ _dl_md_reloc(elf_object_t *object, int r
> > #endif
> >
> > switch (type) {
> > - case RELOC_DIR32:
> > + case R_TYPE(DIR32):
> > if (ELF_R_SYM(rela->r_info) && sym->st_name) {
> > *pt = sr.obj->obj_base + sr.sym->st_value +
> > rela->r_addend;
> > @@ -213,7 +213,7 @@ _dl_md_reloc(elf_object_t *object, int r
> > }
> > break;
> >
> > - case RELOC_PLABEL32:
> > + case R_TYPE(PLABEL32):
> > if (ELF_R_SYM(rela->r_info)) {
> > if (ELF_ST_TYPE(sr.sym->st_info) !=
> STT_FUNC) {
> > DL_DEB(("[%x]PLABEL32: bad\n", i));
> > @@ -236,7 +236,7 @@ _dl_md_reloc(elf_object_t *object, int r
> > }
> > break;
> >
> > - case RELOC_IPLT:
> > + case R_TYPE(IPLT):
> > if (ELF_R_SYM(rela->r_info)) {
> > pt[0] = sr.obj->obj_base +
> sr.sym->st_value +
> > rela->r_addend;
> > @@ -256,7 +256,7 @@ _dl_md_reloc(elf_object_t *object, int r
> > }
> > break;
> >
> > - case RELOC_COPY:
> > + case R_TYPE(COPY):
> > {
> > sr = _dl_find_symbol(symn,
> > SYM_SEARCH_OTHER|SYM_WARNNOTFOUND|SYM_NOTPLT,
> > @@ -381,7 +381,7 @@ _dl_md_reloc_got(elf_object_t *object, i
> > for (i = 0; i < numrela; i++, rela++) {
> > Elf_Addr *r_addr = (Elf_Addr *)(ooff +
> rela->r_offset);
> >
> > - if (ELF_R_TYPE(rela->r_info) != RELOC_IPLT) {
> > + if (ELF_R_TYPE(rela->r_info) != R_TYPE(IPLT)) {
> > _dl_printf("unexpected reloc 0x%x\n",
> > ELF_R_TYPE(rela->r_info));
> > return 1;
> > Index: libexec/ld.so/m88k/rtld_machine.c
> > ===
> > RCS file: /cvs/src/libexec/ld.so/m88k/rtld_machine.c,v
> > retrieving revision 1.31
> > diff -u -p -r1.31 rtld_machine.c
> > --- libexec/ld.so/m88k/rtld_machine.c 8 Jan 2022 06:49:42 -
> 1.31
> > +++ libexec/ld.so/m88k/rtld_machine.c 9 Jan 2022 22:03:27 -
> > @@ -99,17 +99,17 @@ _dl_md_reloc(elf_object_t *object, int r
> >
> > type = ELF_R_TYPE(relas->r_info);
> >
> > - if (type == RELOC_GOTP_ENT && rel != DT_JMPREL)
> > + if (type == R_TYPE(GOTP_ENT) && rel != DT_JMPREL)
> > continue;
> >
> > - if (type == RELOC_NONE)
> > +
Re: WTERMSIG behavior difference
On Thu, Jan 6, 2022 at 10:47 PM Greg Steuck wrote: > I started by debugging a weird test failure in ghc. I narrowed it > down to a simple C program which behaves differently between OpenBSD and > FreeBSD. I stared at the headers of both systems for a bit and still > don't see why on OpenBSD the program prints: > > uname -a ; cc ./a.c && ./a.out > OpenBSD home.nest.cx 7.0 GENERIC.MP#28 amd64 > x 0 pret 130 > > Yet on FreeBSD: > uname -a ; cc ./a.c && ./a.out > FreeBSD ... 13.0-STABLE FreeBSD 13.0-STABLE > x 0 pret -2 > > Where's the signed/unsigned confusion hiding? > No where. The difference in behavior is that of 'sh' when signaled. Run your test programs under "ktrace -i" and compare the behavioral difference of the child 'sh' process after reception of the SIGINT. Philip Guenther
Re: less: fix use after free bug
On Fri, Dec 31, 2021 at 6:22 AM Tobias Stoeckmann
wrote:
> Hi,
>
> it is possible to trigger a use after free bug in less with huge
> files or tight memory constraints. PoC with 100 MB file:
>
> dd if=/dev/zero bs=1024 count=102400 | tr '\0' 'a' > less-poc.txt
> ulimit -d 157286
> less less-poc.txt
>
> The linebuf and attr buffers in line.c are supposed to never be freed,
> since they are used for keeping (meta) data of current line in RAM.
>
> The linebuf and attr buffers are expanded in expand_linebuf. If linebuf
> could be expanded but attr could not, then returned pointers are freed,
> but linebuf variable still points to previous location. Subsequent
> accesses will therefore trigger a use after free bug.
>
> The easiest fix is to keep reallocated linebuf. The size of both buffers
> differ, but the code still assumes that the previous (smaller) size is
> the correct one.
>
> Thoughts on this approach?
>
Analysis makes sense.
To bikeshed slightly I would be inclined to do the work progressively,
perhaps like the diff below...but your diff works too.
Philip Guenther
Index: line.c
===
RCS file: /data/src/openbsd/src/usr.bin/less/line.c,v
retrieving revision 1.34
diff -u -p -r1.34 line.c
--- line.c 25 Oct 2021 19:54:29 - 1.34
+++ line.c 1 Jan 2022 06:24:31 -
@@ -96,16 +96,16 @@ expand_linebuf(void)
/* Just realloc to expand the buffer, if we can. */
char *new_buf = recallocarray(linebuf, size_linebuf, new_size, 1);
- char *new_attr = recallocarray(attr, size_linebuf, new_size, 1);
- if (new_buf == NULL || new_attr == NULL) {
- free(new_attr);
- free(new_buf);
- return (1);
+ if (new_buf != NULL) {
+ char *new_attr = recallocarray(attr, size_linebuf,
new_size, 1);
+ linebuf = new_buf;
+ if (new_attr != NULL) {
+ attr = new_attr;
+ size_linebuf = new_size;
+ return (0);
+ }
}
- linebuf = new_buf;
- attr = new_attr;
- size_linebuf = new_size;
- return (0);
+ return (1);
}
/*
Re: Fix GNUism in bsd.dep.mk
On Fri, Dec 31, 2021 at 7:44 AM Christian Ehrhardt
wrote:
> Here at genua, trying to build libpcap sometimes breaks in
> libpcap with the following error message:
>
> | Using $< in a non-suffix rule context is a GNUmake idiom \
> |(/data/git/ehrhardt/genuos/os/mk/bsd.dep.mk:47)
>
> The bug is in bsd.dep.mk where ${.IMPSRC} (aka $<) is used
> in a context where it is not neccessarily defined by OpenBSD make.
>
> Below is a diff to Use ${.ALLSRC:M*.y} instead of ${.IMPSRC} as
> the input file for yacc.
>
> The issue is with the rule for the grammar.h file that is generated
> by yacc from grammar.c. You can easily reproduce the bug with the
> following steps:
> - build libpcap from scratch: cd .../src/lib/libpcap && make clean all
> - remove the generated grammar.h file: rm obj*/grammar.h
> - build libpcap again (incremental build): make
> In normal builds this does not trigger as grammar.h is implicitly
> generated by the rule for grammar.c and when make checks for
> dependencies it simply finds grammar.h uptodate. However,
> incremental or parallel builds might decide to make grammar.h
> from grammar.y.
>
> Now, why is this only a problem for grammar.h but not for
> grammar.c? The answer to this question is burried deeply in
> OpenBSD's mk files.
>
> The snippet in bsd.dep.mk that triggers the error is a single
> rule statement that generates foo.c and foo.h from foo.y with a
> call to yacc -d. The rule is generated with a loop, i.e. it is not
> a prefix rule. However, a prefix rule context is required for
> the use of ${.IMPSRC} aka $<. For the .c file such a prefix
> rule is provided by bsd.sys.mk and this rule is in scope when
> make evaluates the yacc rule. However, for .h file generation
> from a .y file there is no such prefix rule defined in any of
> the Makefiles. Even if it were the .h suffix is missing from
> .SUFFIXES and the rule would not be considered.
>
> NOTE: The obvious way to fix this would be to use $f instead of
> ${.IMPSRC}. However, this does not work as $f is then missing
> the path prefix and yacc won't find it if an obj directory is
> used. This is probably the reason for the use of ${.IMPSRC} in
> the first place.
>
Ah, nice analysis! The suggested fix looks safe to me (can't see how a .c
could have two .y direct prerequisites, so this can't be ambiguous).
ok guenther@
Re: hangman(6): skip retguard symbols
Skipping all leading double-underbar symbols makes sense to me. Tempting to skip all symbols with more than one digit (or maybe just more than one consecutive digit?), as guessing among per-chip symbols from, say, the ar* or dc* families is an exercise in futility. On Fri, Dec 24, 2021 at 5:23 AM Theo Buehler wrote: > __retguard_ symbols are easy to recognize and no real fun to guess. > I suggest we skip them. Perhaps we should even skip all __* symbols? > > Index: ksyms.c > === > RCS file: /cvs/src/games/hangman/ksyms.c,v > retrieving revision 1.12 > diff -u -p -r1.12 ksyms.c > --- ksyms.c 28 Jun 2019 13:32:52 - 1.12 > +++ ksyms.c 23 Dec 2021 18:28:02 - > @@ -70,6 +70,9 @@ sym_getword(void) > /* ignore symbols containing dots or dollar signs */ > if (strchr(sym, '.') != NULL || strchr(sym, '$') != NULL) > continue; > + /* guessing retguard symbols is no fun */ > + if (strncmp(sym, "__retguard", 10) == 0) > + continue; > > break; > } > >
warning for -current builders: update your kernel before libc/ld.so!
For those building -current themselves, when you update past the commit below you must be sure to build *and reboot to* a new kernel with the change before you install a new libc or ld.so! If you fail to do so then anything using the newer-than-kernel libc/ld.so will coredump immediately, generally on the first mmap(2), and you'll need to reboot to a bsd.rd or similar and put a matching kernel+libc+ld.so in place. This might be a good time to just install an official snapshot instead. -- Forwarded message - From: Philip Guenther Date: Thu, Dec 23, 2021 at 10:51 AM Subject: CVS: cvs.openbsd.org: src To: CVSROOT:/cvs Module name:src Changes by: guent...@cvs.openbsd.org2021/12/23 11:50:33 Modified files: sys/kern : syscalls.master vfs_syscalls.c kern_ktrace.c kern_pledge.c sys/arch/sh/sh : trap.c sys/arch/hppa/hppa: trap.c sys/uvm: uvm_mmap.c lib/libc/sys : Makefile.inc libexec/ld.so : Makefile loader.c libexec/ld.so/m88k: rtld_machine.c usr.bin/kdump : kdump.c Added files: libexec/ld.so : syscall.h Removed files: lib/libc/sys : ftruncate.c lseek.c mmap.c mquery.c pread.c preadv.c pwrite.c pwritev.c truncate.c libexec/ld.so/m88k: syscall.h libexec/ld.so/aarch64: syscall.h libexec/ld.so/alpha: syscall.h libexec/ld.so/amd64: syscall.h libexec/ld.so/arm: syscall.h libexec/ld.so/hppa: syscall.h libexec/ld.so/i386: syscall.h libexec/ld.so/mips64: syscall.h libexec/ld.so/powerpc: syscall.h libexec/ld.so/powerpc64: syscall.h libexec/ld.so/riscv64: syscall.h libexec/ld.so/sh: syscall.h libexec/ld.so/sparc64: syscall.h Log message: Roll the syscalls that have an off_t argument to remove the explicit padding. Switch libc and ld.so to the generic stubs for these calls. WARNING: reboot to updated kernel before installing libc or ld.so! Time for a story... When gcc (back in 1.x days) first implemented long long, it didn't (always) pass 64bit arguments in 'aligned' registers/stack slots, with the result that argument offsets didn't match structure offsets. This affected the nine system calls that pass off_t arguments: ftruncate lseek mmap mquery pread preadv pwrite pwritev truncate To avoid having to do custom ASM wrappers for those, BSD put an explicit pad argument in so that the off_t argument would always start on a even slot and thus be naturally aligned. Thus those odd wrappers in lib/libc/sys/ that use __syscall() and pass an extra '0' argument. The ABIs for different CPUs eventually settled how things should be passed on each and gcc 2.x followed them. The only arch now where it helps is landisk, which needs to skip the last argument register if it would be the first half of a 64bit argument. So: add new syscalls without the pad argument and on landisk do that skipping directly in the syscall handler in the kernel. Keep compat support for the existing syscalls long enough for the transition. ok deraadt@
Re: moncontrol(3): remove hertz()
On Mon, Dec 6, 2021 at 10:30 AM Scott Cheloha wrote: > In the moncontrol(3) code we have this fallback function, hertz(). > The idea is to use an undocumented side effect of setitimer(2) to > determine the width of the hardclock(9) tick. > > hertz() has a number of problems: > > So, I want to get rid of hertz() and replace it with nothing. This is > an extreme edge case. sysctl(2) has to fail. I do not think that is > possible here, outside of stack corruption. > I agree: sysctl(KERN_CLOCKRATE) isn't blocked by pledge(2), so how would they manage to break it? (I'm not even sure the failover from profhz to hz should be kept: the kernel does that itself in initclocks(9) so if profhz is 0 then hz is too, sysctl(2) failed, and your CPU is on fire.) ok guenther@
Re: New hw.perfpolicy behavior
On Tue, Nov 9, 2021 at 3:29 PM Jan Stary wrote: > On Nov 10 00:21:59, h...@stare.cz wrote: > > Regarding C states, this machine has > > > > acpicpu0 at acpi0: C3(350@96 mwait.1@0x20), C2(500@64 mwait.1@0x10), > C1(1000@1 mwait.1), PSS > > > > I suppose the cpu supports C1, C2, C3, but can someone please kindly > > explain (or point to an explanation of) what the whole line says? > > For comparison, my Thinkpad T400 has > acpicpu0 at acpi0: !C3(100@57 mwait.3@0x30), !C2(500@1 mwait.1@0x10), > C1(1000@1 mwait.1), PSS > What does the ! mean? > This is generated by sys/dev/acpi/acpicpu.c:acpicpu_print_one_cst() and at this point should either be suppressed by default or documented in the manpage; I'm not sure which. To enumerate it from right to left: !C3(100@57 mwait.3@0x30) ^ ^ ^ ^ ^ ^ ^ | ||| | | | | ||| | | \-- 'hints' passed to the actual MWAIT instruction, or address for 'io' method | ||| | \-- fixed-function level bit flags: 1="HW coordinated" 2="bus-master avoidance required" | ||| | OR '!' if this is a faked-up fallback to the pre-CST C1-via-HALT (bios is broken) | ||| \-- sleep method, one of mwait, io, or halt | ||\-- advertised latency when exiting the state | |\-- advertised power-consumption | \-- the C state, duh \-- '!' to indicate this state is disabled because the CPU's LAPIC stops in C2 or deeper (no ARAT)
Re: [PATCH] /etc/netstart: unquoted command substitution inside arithmetic expression
On Sun, Oct 10, 2021 at 1:48 PM bm1les wrote: > Exactly my point. Even if the circumstances were contrived, I think it > would good to fix it just for the sake of correctness. > Sure, knowing what circumstances could cause a problem assists in achieving correctness. > The issue is actually a pattern I found not only in /etc/netstart but also > in /etc/rc. (( )) cannot deal with an empty result yet it sometimes > includes calls to sysctl which apparently can return an empty string in > some cases. > > So options are: > > 1. ensure that sysctl always returns a number where it is expected > 2. work around the issue by using /bin/[ in place of or in addition to, > the arithmetic expression (depending on the expression being tested), so > that whatever returns empty string can be tested instead of blowing up. > 3. use syntax with works when the expansion is empty, such as (( $(sysctl blah) + 0 != 0 )) which is unary plus when it's empty, or (( $(sysctl blah)0 != 0 )) which multiplies the value by 10 when not empty and is zero when it's empty. Philip Guenther (Per POSIX rules, any arithmetic expression is effectively evaluated in double-quotes. If you follow the spec from there, you can work out that if an arithmetic expression that directly** contains double-quotes parses correctly, then it must also parse correctly with the double-quotes removed; adding double-quotes can't fix an arithmetic expression. The reverse is not true: adding double-quotes can break the parsing. ** i.e., not counting quotes inside a nested parameter expansion or command substitution.)
Re: [PATCH] /etc/netstart: unquoted command substitution inside arithmetic expression
On Fri, Oct 8, 2021 at 8:57 AM Theo de Raadt wrote: > Philip Guenther wrote: > > > On Thu, Oct 7, 2021 at 5:57 PM bm1les wrote: > > > > > --- netstart2 Sep 2021 19:38:20 - 1.216 > > > +++ netstart8 Oct 2021 02:43:30 - > > > @@ -365,7 +365,7 @@ ifmstart "tun tap gif etherip gre egre p > > > if [[ $ip6kernel == YES ]]; then > > > # Ensure IPv6 Duplicate Address Detection (DAD) is completed. > > > count=0 > > > - while ((count++ < 10 && $(sysctl -n net.inet6.ip6.dad_pending) > != > > > 0)); do > > > + while ((count++ < 10 && "$(sysctl -n > net.inet6.ip6.dad_pending)" > > > != 0)); do > > > sleep 1 > > > done > > > fi > > > > > > > I can't figure out what problem you think this could solve. Can you > > explain the circumstances under which those quotes could make a > difference? > > Not the OP's issue, but I think a kernels compiled without option INET6 > will return an errno, and I cannot tell if sysctl prints out an error > message > or converts to "", the empty string, which would conceivably mis-parse. > AFAICT, an empty quoted string there results in the exact same error. As I wrote off-list to the original submitter: Can you be clearer about how the quoting makes the result any better when > run under bsd.rd? Doesn't it fail in the same way? Testing with 'echo' > instead would seem to indicate so: > : bleys; (( 1 < 10 && $(echo) != 0 )); echo $? > /bin/ksh: 1 < 10 && != 0 : unexpected `!=' > 2 > : bleys; (( 1 < 10 && $(echo -n) != 0 )); echo $? > /bin/ksh: 1 < 10 && != 0 : unexpected `!=' > 2 > : bleys; (( 1 < 10 && "$(echo)" != 0 )); echo $? > /bin/ksh: 1 < 10 && != 0 : unexpected `!=' > 2 > : bleys; (( 1 < 10 && "$(echo -n)" != 0 )); echo $? > /bin/ksh: 1 < 10 && != 0 : unexpected `!=' > 2 > : bleys; Philip
Re: [PATCH] /etc/netstart: unquoted command substitution inside arithmetic expression
On Thu, Oct 7, 2021 at 5:57 PM bm1les wrote: > --- netstart2 Sep 2021 19:38:20 - 1.216 > +++ netstart8 Oct 2021 02:43:30 - > @@ -365,7 +365,7 @@ ifmstart "tun tap gif etherip gre egre p > if [[ $ip6kernel == YES ]]; then > # Ensure IPv6 Duplicate Address Detection (DAD) is completed. > count=0 > - while ((count++ < 10 && $(sysctl -n net.inet6.ip6.dad_pending) != > 0)); do > + while ((count++ < 10 && "$(sysctl -n net.inet6.ip6.dad_pending)" > != 0)); do > sleep 1 > done > fi > I can't figure out what problem you think this could solve. Can you explain the circumstances under which those quotes could make a difference? Philip Guenther
Re: mi_switch() & setting `p_stat'
On Sat, Oct 2, 2021 at 8:57 AM Martin Pieuchot wrote: > When a thread running on a CPU schedules itself out, it does the following > (pseudo_code): > > SCHED_LOCK() > curproc->p_stat = SSLEEP; > // some more operations > mi_switch() > > The problem with this is that any instrumentation between setting `p_stat' > and cpu_switchto() is incorrect because 'curproc' is still being executed > and is not yet sleeping. Its `p_stat' should be SONPROC and not SSLEEP. > ... > To fix this we should set `p_stat' as late a possible, diff below does that > just before calling cpu_switchto(). > ... > --- kern/kern_sig.c 28 Sep 2021 10:00:18 - 1.283 > +++ kern/kern_sig.c 2 Oct 2021 17:00:52 - > @@ -1347,7 +1347,6 @@ proc_stop(struct proc *p, int sw) > SCHED_ASSERT_LOCKED(); > #endif > > - p->p_stat = SSTOP; > atomic_clearbits_int(>ps_flags, PS_WAITED); > atomic_setbits_int(>ps_flags, PS_STOPPED); > atomic_setbits_int(>p_flag, P_SUSPSIG); > @@ -1357,7 +1356,7 @@ proc_stop(struct proc *p, int sw) > */ > softintr_schedule(proc_stop_si); > if (sw) > - mi_switch(); > + mi_switch(SSTOP); > This pair of chunks is wrong, as then the proc_stop(p, 0) call in ptsignal() doesn't change the process from SSLEEP to SSTOP. Sending a stop signal to a blocked process needs to change its state. Philip Guenther
Re: rsync fix symlink discovery
Based on the fts_open(3) manpage and other base source usage, shouldn't
this use fts_accpath instead of fts_name?
The use of fts_statp in this code seems a bit loose vs ftp_info: instead of
using S_ISLNK() on fts_statp I would expect this code to check for fts_info
== FTS_SL: according the manpage fts_statp's value is undefined for various
values of fts_info.
Philip Guenther
On Fri, Jul 2, 2021 at 4:46 AM Claudio Jeker
wrote:
> Hit this today while doing some tests. symlink_read() needs to use just
> the filename and not the full path because fts_read(3) does chdir
> internally.
>
> Without this I got:
> openrsync: error: ./obj/openrsync.1: readlink: No such file or directory
> openrsync: error: symlink_read
> openrsync: error: flist_gen_dirent
> openrsync: error: flist_gen
> openrsync: error: rsync_sender
>
> --
> :wq Claudio
>
> Index: flist.c
> ===
> RCS file: /cvs/src/usr.bin/rsync/flist.c,v
> retrieving revision 1.32
> diff -u -p -r1.32 flist.c
> --- flist.c 30 Jun 2021 13:10:04 - 1.32
> +++ flist.c 2 Jul 2021 13:14:01 -
> @@ -972,7 +972,7 @@ flist_gen_dirent(struct sess *sess, char
> /* Optionally copy link information. */
>
> if (S_ISLNK(ent->fts_statp->st_mode)) {
> - f->link = symlink_read(f->path);
> + f->link = symlink_read(ent->fts_name);
> if (f->link == NULL) {
> ERRX1("symlink_read");
> goto out;
>
>
Re: ld.so: program headers: do not rely on DYNAMIC coming before GNU_RELRO
On Mon, May 24, 2021 at 4:59 AM Klemens Nanni wrote:
> When tinkering with ld.so crashes due to file corruption the other day
> I tested a few changes but did not want to replace /usr/libexec/ld.so
> and since recompiling executable to change their interpreter is not
> always an option, I went for https://github.com/NixOS/patchelf which
> allows me to manipulate executables in place.
>
> The tool works just fine and as a byproduct rearanges program headers;
> that in itself is fine except our run-time link-editor is not happy with
> such valid executables:
>
>
> ELF mandates nothing but the file header be at a fixed location, hence
> ld.so(1) must not assume any specific order for headers, segments, etc.
>
(Not quite: it does place ordering requirements in some specific cases,
such as that PT_INTERP and PT_PHDR, if present, must appear before any
PT_LOAD segments in the program header, and that PT_LOAD segments must
appear in p_vaddr order.)
Looping over the program header table to parse segment headers,
> _dl_boot() creates the executable object upon DYNAMIC and expects it to
> be set upon GNU_RELRO, resulting in a NULL dereference iff that order is
> reversed.
>
> Store relocation bits in temporary variables and update the executable
> object once all segment headers are parsed to lift this dependency.
>
> Under __mips__ _dl_boot() later on uses the same temporary variable, so
> move nothing but the declaration out of MI code so as to not alter the
> MD code's logic/behaviour.
>
>
> This fix is needed for the following work on OpenBSD:
>
> $ patchelf --set-interpreter $PWD/obj/ld.so ./my-ldso-test
> $ readelf -l ./my-ldso-test | grep interpreter
> [Requesting program interpreter: /usr/src/libexec/
> ld.so/obj/ld.so]
> $ ./my-ldso-test
> it works!
>
> amd64 and arm64 regress is happy and all my patched executables work
> with this.
>
> Feedback? Objections? OK?
>
> diff d7231fb4fb547dd287a884c56ae7c8b10f9145fe
> f023dbe355bef379d55eb93eddbb2702559d5bdb
> blob - 18bd30af759bffbc4e3fbfee9ffc29906f0d1bb0
> blob + b66dbb169aad9afffa1283d480ad9276aff9072a
> --- libexec/ld.so/loader.c
> +++ libexec/ld.so/loader.c
> @@ -464,6 +464,7 @@ _dl_boot(const char **argv, char **envp, const long dy
> int failed;
> struct dep_node *n;
> Elf_Addr minva, maxva, exe_loff, exec_end, cur_exec_end;
> + Elf_Addr relro_addr = 0, relro_size = 0;
> Elf_Phdr *ptls = NULL;
> int align;
>
> @@ -552,8 +553,8 @@ _dl_boot(const char **argv, char **envp, const long dy
> ptls = phdp;
> break;
> case PT_GNU_RELRO:
> - exe_obj->relro_addr = phdp->p_vaddr + exe_loff;
> - exe_obj->relro_size = phdp->p_memsz;
> + relro_addr = phdp->p_vaddr + exe_loff;
> + relro_size = phdp->p_memsz;
> break;
> }
> phdp++;
> @@ -561,6 +562,8 @@ _dl_boot(const char **argv, char **envp, const long dy
> exe_obj->load_list = load_list;
> exe_obj->obj_flags |= DF_1_GLOBAL;
> exe_obj->load_size = maxva - minva;
> + exe_obj->relro_addr = relro_addr;
> + exe_obj->relro_size = relro_size;
> _dl_set_sod(exe_obj->load_name, _obj->sod);
>
> #ifdef __i386__
> @@ -638,7 +641,7 @@ _dl_boot(const char **argv, char **envp, const long dy
> debug_map->r_ldbase = dyn_loff;
> _dl_debug_map = debug_map;
> #ifdef __mips__
> - Elf_Addr relro_addr = exe_obj->relro_addr;
> + relro_addr = exe_obj->relro_addr;
> if (dynp->d_tag == DT_DEBUG &&
> ((Elf_Addr)map_link + sizeof(*map_link) <= relro_addr
> ||
> (Elf_Addr)map_link >= relro_addr +
> exe_obj->relro_size)) {
>
>
ok guenther@
(This still assumes PT_GNU_RELRO comes after PT_PHDR, for exe_loff, but I
suspect even patchelf wouldn't screw with that.)
Re: Use atomic op for UVM map refcount
On Wed, May 19, 2021 at 11:29 PM Martin Pieuchot wrote: > On 19/05/21(Wed) 16:17, Mark Kettenis wrote: > ... > > There are the READ_ONCE() and WRITE_ONCE() macros. I'm not a big fan > > of those (since they add clutter) but they do take care of dependency > > ordering issues that exist in the alpha memory model. Must admit that > > I only vaguely understand that issue, but I think it involves ordered > > access to two atomic variables which doesn't seem to be the case. > > These macros are used in places where declaring the field as "volatile" > could also work, no? We can look at __mp_lock and SMR implementations. > So could we agree one way to do things? > > Visa, David, why did you pick READ_ONCE() in SMR and veb(4)? Anything > we overlooked regarding the use of "volatile"? > If _all_ references to a member/variable use READ/WRITE_ONCE, then declaring it volatile should be equivalent, but if there are any uses which have a lock for protection instead than making the member volatile will result in worse object code for the protected sequences where *_ONCE are not needed. Also, initialization of structs with volatile members can be pessimal: the compiler has to assume this could be in uncached memory mapped from a device where writes to the member have to be of the size of the member: no paving with larger writes or deferring initializations. volatile is a *really* blunt hammer. READ/WRITE_ONCE use it carefully to build a sharper tool. Unifying on "just plain volatile" when the work has already been done to use a sharper tool correctly..well, if that's a good idea then why have SMR at all when locks would be easier for everyone to think about, despite being a blunter hammer? /s Philip Guenther
Re: Use atomic op for UVM map refcount
On Wed, May 19, 2021 at 5:19 AM Mark Kettenis wrote: > > Date: Tue, 18 May 2021 13:24:42 +0200 > > From: Martin Pieuchot > ... > > There's only a couple of 'volatile' usages in sys/sys. These annotations > > do not explicitly indicate which piece of code requires it. Maybe it > would > > be clearer to use a cast or a macro where necessary. This might help us > > understand why and where "volatile" is needed. > > There are the READ_ONCE() and WRITE_ONCE() macros. I'm not a big fan > of those (since they add clutter) but they do take care of dependency > ordering issues that exist in the alpha memory model. Must admit that > I only vaguely understand that issue, but I think it involves ordered > access to two atomic variables which doesn't seem to be the case. > > On non-alpha systems, READ_ONCE() and WRITE_ONCE() just do a volatile > pointer cast. > READ/WRITE_ONCE() are 99% about keeping the compiler from deciding to rearrange code such that the indicated variable is read/written more than once. To point to Linus posts from 2008/2009, when it was still ACCESS_ONCE(): https://yarchive.net/comp/linux/ACCESS_ONCE.html ISTR a paper by Ousterhout describing this same problem to the C standard committee(?) in the early 90's, which kinda opened the "memory model" rathole. If the variable is actually being protected by a lock then these are indeed noise/pessimization; it's the lock-less accesses where the compiler can pull a rabbit from its hat and stab you with it. Philip Guenther
Re: ld.so: NULL dereference on corrupt library
On Tue, May 4, 2021 at 12:43 PM Klemens Nanni wrote: ... > I compared my corrupted shared library with an intact copy from ports > and it showed that the corrupted one was simply zeroed out at some point > (not truncated) until the end, e.g. readelf(1)'s `-h' or `-l' report > "Error: no .dynamic section in the dynamic segment". > > So this isn't a case of some badly linked library or one that has a few > bits flipped, it's simply a partial one... seems like bad luck? > IMHO, the benefit of adding this check is almost zero: it gives a slightly better experience for a small set of possible data corruption cases, when similar corruptions that affect other pages aren't helped at all as it'll crash when it executes zeroed text, or accesses zeroed data, or fails to find a required symbol because the symbol table was zeroed out. If we want to protect against that sort of hardware lossage, then a filesystem which does so is the way to go, not an alarm on one window of a glass house. Philip Guenther
Re: printf(1) Fix hex numbers
On Sun, Apr 18, 2021 at 12:04 PM Martijn van Duren < openbsd+t...@list.imperialat.at> wrote: > On Sun, 2021-04-18 at 11:17 -0900, Philip Guenther wrote: > > > > I'll just throw in a note that the current POSIX spec does not include > support for \x in the printf(1) format or in the argument to the %b format > specifier. At least for \x in the format string it > > appears to require that it _not_ be interpreted, such that > > printf '\x61\n' > > must output > > \x61 > > and not > > a > > Could you point out where it states that an escape-sequence not > specified must output the original text? The way I read it[0][1]: > > In addition to the escape sequences shown in XBD File Format Notation ( > '\\', '\a', '\b', '\f', '\n', '\r', '\t', '\v' ), "\ddd", where ddd is a > one, two, or three-digit octal number, shall be written as a byte with > the numeric value specified by the octal number. > > and > > Escape Sequences and Associated Actions lists escape sequences and > associated actions on display devices capable of the action. > Before that statement in XBD 5 is this: - The format is a character string that contains three types of objects defined below: 1. Characters that are not "escape sequences" or "conversion specifications", as described below, shall be copied to the output. 2. Escape Sequences represent non-graphic characters and the escape character (). 3. Conversion Specifications specify the output format of each argument; see below. - So, per (1) above, if it's not an escape sequence or conversion specification, it SHALL be copied to the output. Unfortunately, the list of escape sequences is just a table, and while the printf(1) specification has the clause you cited that adds \ddd to the list, there's nothing to state that, for formats, followed by anything else results in unspecified behavior. Ergo \x isn't an escape sequence under (2) and, since it's not a converspecification, it must follow (1) and be copied to the output. Anyway, POSIX wording will probably change, it's just a place to note in the manual that handling \xXX is an extension to the POSIX standard and that compliant (or portable) applications must use \ddd instead. Philip Guenther
Re: printf(1) Fix hex numbers
I'll just throw in a note that the current POSIX spec does not include support for \x in the printf(1) format or in the argument to the %b format specifier. At least for \x in the format string it appears to require that it _not_ be interpreted, such that printf '\x61\n' must output \x61 and not a FreeBSD 12's /usr/bin/printf does this, for example, however at least some on the list seemed to feel the spec should be changed to make it unspecified behavior. The behavior of printf '%b\n' '\x61' is clearly unspecified already. Philip Guenther On Sun, Apr 18, 2021 at 7:02 AM Martijn van Duren < openbsd+t...@list.imperialat.at> wrote: > On Thu, 2021-04-15 at 19:48 -0600, Todd C. Miller wrote: > > On Fri, 16 Apr 2021 03:34:04 +0200, Martijn van Duren wrote: > > > > > We currently have NetBSD's behaviour when it comes to to no characters, > > > so no need to change there imho, but the 2 character limit seems like > > > a good place to stop, especially since there's no way to break out of > > > this hungry hungry hippo. > > > > > > limiter inspiration taken from octal notation. > > > > > > While here, also document \x notation. > > > > No objection from me but I'm not sure it needs to make 6.9. > > > > - todd > I found a (different) bug in NetBSD's implementation, which got fixed by > christos. But he also added a check that at least 1 isxdigit(3) needs to > be present. From the commit message: > Also add a warning if the conversion fails (like the gnu printf does) > > Do we want something similar now that we are the odd ones out, or keep > the diff as is? I still think that the current diff is fine, but since > the landscape changed I wanted to throw it out there. > > martijn@ > >
Re: rw_lock_held() & friends
On Mon, Dec 7, 2020 at 11:16 AM Alexandr Nedvedicky < alexandr.nedvedi...@oracle.com> wrote: > What's the plan for rw_write_held()? Currently macro is true, if whoever > holds > the lock. > > > > > +#define rw_write_held(rwl) (rw_status(rwl) == RW_WRITE) > Nope. rw_status() returns RW_WRITE_OTHER if a different thread holds the lock. Philip
Re: Use SMR_TAILQ for `ps_threads'
On Tue, Dec 1, 2020 at 5:48 AM Martin Pieuchot wrote: ... > exit1() for a multi-threaded process does the following: > > atomic_setbits_int(>p_flag, P_WEXIT); > single_thread_check(p, 0); // directly or via single_thread_set() > SMR_TAILQ_REMOVE_LOCKED(>ps_threads, p, p_thr_link); > > Note that exit1() can't be called in parallel. > Well, multiple threads in a process can easily hit exit1() practically concurrently, under the limitations of the kernel lock. If anything in exit1() sleeps, it has to be ready for some other thread to start into exit1()! If exit1() is called with EXIT_NORMAL, the current thread will start by > setting `ps_single' then iterate over `ps_thread'. This is done before > updating `ps_thread' so there's no race in that case. > > If exit1() is called with EXIT_THREAD, the current thread might end up > removing itself from `ps_thread' while another one is iterating over it. > Since we're currently concerned about the iteration in single_thread_set() > notice that `P_WEXIT' is checked first. So if my understanding is correct > is should be enough to do the following: > > SMR_TAILQ_REMOVE_LOCKED(>ps_threads, p, p_thr_link); > smr_barrier(); > > That would delays exit1() a bit but doesn't involve callback which is > IMHO simpler. > > Did I miss anything? > What does it take to be sure that an "is empty" (or "only contains one") test is _really_ true with SMR? Philip
Re: stdio: fclose(3), fopen(3): intended locking hierarchy?
On Mon, Nov 30, 2020 at 6:10 PM Scott Cheloha wrote: > On Sat, Nov 28, 2020 at 01:41:50PM -0800, Philip Guenther wrote: > ... > > After thinking through states more, #4 isn't safe: _fwalk() can't take > > sfp_mutex because it's called from __srefill() which has its FILE locked, > > which would reverse the order: two concurrent calls to __srefill() from > > line-buffered FILEs could have one in _fwalk() blocking on locking the > > other, while the other blocks on the sfp_mutex for _fwalk(). > > This part in __srefill(): > > /* > * Before reading from a line buffered or unbuffered file, > * flush all line buffered output files, per the ANSI C > * standard. > */ > ... > Where in the standard(s) is this behavior required? I'm not even sure > how to look for it. > Pick a version of the C standard and search for "buffered" until you find something like this, which is part of 7.19.3p3 in the draft I'm looking at: <...> When a stream is line buffered, characters are intended to be transmitted to or from the host environment as a block when a new-line character is encountered. Furthermore, characters are intended to be transmitted as a block to the host environment when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line buffered stream that requires the transmission of characters from the host environment. Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions. Effectively the same text appears in the POSIX standard in XSH 2.5p6. Basically, you're allowed to stop doing that by instead printing out your cell-phone number so that everyone who wants to complain that their program failed to output its prompt before blocking for input can call and scream at you. :D Philip Guenther
Re: kvm_getfiles and KERN_FILE_BYFILE
On Sun, Nov 29, 2020 at 12:14 PM Martijn van Duren < openbsd+t...@list.imperialat.at> wrote: > On Sat, 2020-11-28 at 16:23 -0800, Philip Guenther wrote: > > On Thu, Nov 26, 2020 at 1:08 PM Martijn van Duren < > openbsd+t...@list.imperialat.at> wrote: > > > I'm currently playing around a bit with kvm_getfiles and found that I > > > couldn't use KERN_FILE_BYFILE with DTYPE_SOCKET. > > > According to kvm_getfiles(3): > > > For KERN_FILE_BYFILE the recognized file types are defined in > > > : > > > > > >DTYPE_VNODE files and devices > > >DTYPE_SOCKET sockets, regardless of domain > > >DTYPE_PIPEpipes and FIFOs > > >DTYPE_KQUEUE kqueues > > > > > > But these defines are under ifdef _KERNEL. > > > > > > So is the manpage lying here, or should the defines be hoisted out > > > of the ifdef? > > > > > > > > > Let's go ahead and hoist them: FreeBSD and NetBSD already have. If > possible, the diff to do that should also simplify the #include bits in > these files: > > usr.bin/netstat/inet.c > > usr.bin/fstat/fstat.c > > usr.bin/fstat/fuser.c > > usr.bin/systat/netstat.c > > > > > > Philip Guenther > > > > The others have the #endif/#ifdef break rather low in the file. > Personally I reckon it's better reading if the common code is more > towards the top. > > OK? > ok guenther@ How do the userland clean up bits look? Philip Guenther
Re: kvm_getfiles and KERN_FILE_BYFILE
On Thu, Nov 26, 2020 at 1:08 PM Martijn van Duren <
openbsd+t...@list.imperialat.at> wrote:
> I'm currently playing around a bit with kvm_getfiles and found that I
> couldn't use KERN_FILE_BYFILE with DTYPE_SOCKET.
> According to kvm_getfiles(3):
> For KERN_FILE_BYFILE the recognized file types are defined in
> :
>
>DTYPE_VNODE files and devices
>DTYPE_SOCKET sockets, regardless of domain
>DTYPE_PIPEpipes and FIFOs
>DTYPE_KQUEUE kqueues
>
> But these defines are under ifdef _KERNEL.
>
> So is the manpage lying here, or should the defines be hoisted out
> of the ifdef?
>
Let's go ahead and hoist them: FreeBSD and NetBSD already have. If
possible, the diff to do that should also simplify the #include bits in
these files:
usr.bin/netstat/inet.c
usr.bin/fstat/fstat.c
usr.bin/fstat/fuser.c
usr.bin/systat/netstat.c
Philip Guenther
Re: stdio: fclose(3), fopen(3): intended locking hierarchy?
On Fri, Nov 27, 2020 at 10:35 PM Philip Guenther wrote:
...
> So yeah, maybe it does work to:
> 1) make __sfp() FLOCKFILE() the allocated FILE before unlocking sfp_mutex
> 2) update f{,d,mem,un}open() and open_*memstream() to match (1), unlocking
>in all paths when the FILE is safe to be accessed by _fwalk(), and
> locking
>sfp_mutex around the zeroing of _flags.
> 3) make fclose() and freopen() also lock sfp_mutex around the zeroing of
> _flags
>(should add an _frelease() to findfp.c that does this dance for (2) and
> (3))
> 4) make _fwalk() take sfp_mutex, and maybe also a FILE* so the setting of
>__SIGN can be done under the lock?
>
5) decide how/whether to handle adjust the FLOCKFILE placement in the
> _fwalk()
>tree: is the testing of the "is line-buffered" flag in lflush() safe
> without
>a lock? Mumble...
>
After thinking through states more, #4 isn't safe: _fwalk() can't take
sfp_mutex because it's called from __srefill() which has its FILE locked,
which would reverse the order: two concurrent calls to __srefill() from
line-buffered FILEs could have one in _fwalk() blocking on locking the
other, while the other blocks on the sfp_mutex for _fwalk().
Hmm, there's currently a loop between two __srefill() calls like that, as
there's nothing to force visibility of the __SIGN flag between CPUs so they
could try to lock each other. Grrr.
Time to check other BSDs and see if they have a good solution to this...
Philip
>
> Now, back to that first assumption: if you're not willing to assume
> it then the "is allocated" test needs to not use _flags but be some
> other state change which can be relied on to not have false-positives,
> but otherwise the other bits above still apply, I believe. Would
> be a major ABI change...and if we do that there's like 3 other
> things we should do at the same time (merge __sfileext into FILE,
> grow _file to an int, and maybe move the recursive mutex for
> f{,un}lockfile() into FILE?)
>
>
> Philip Guenther
>
>
Re: stdio: fclose(3), fopen(3): intended locking hierarchy?
ted" test needs to not use _flags but be some
other state change which can be relied on to not have false-positives,
but otherwise the other bits above still apply, I believe. Would
be a major ABI change...and if we do that there's like 3 other
things we should do at the same time (merge __sfileext into FILE,
grow _file to an int, and maybe move the recursive mutex for
f{,un}lockfile() into FILE?)
Philip Guenther
Re: Fix ilogb(3)
On Fri, Nov 6, 2020 at 4:51 PM George Koehler wrote: > Your ilogb fix is ok gkoehler@ > It's annoying that C and/or ieee754 and the original hardware implementation in the x87 instructions diverged in their definitions, but the former is what matters and libm needs to follow that. ok guenther@ > On Sat, 31 Oct 2020 16:09:07 +0100 (CET) > Mark Kettenis wrote: > > > - Dropping the amd64 and i386 versions. Fixing the corner cases in > > assembly is hard, and the C implementation should be fast enough for > > regular floating-point values. > > The amd64 and i386 assembly uses the x87 fxtract instruction. I feel > that x87 instructions are obsolete on amd64. <...> Umm, no? The amd64 ABI defines "long double" as matching the format used by the x87 instructions; a function returning that type returns it in the x87 %st(0) register and a review of libm finds many of the functions are naturally implemented by the native x87 instructions. I believe the main issue in this case is that the standard evolved away from what Kahan originally did as a consultant with Intel. For a note on how the ieee754 standard originated in (large) part from what Kahan did with Intel, see https://people.eecs.berkeley.edu/~wkahan/ieee754status/754story.html I'm not nearly enough of a numerical analyst to judge the decision of the standard. Philip Guenther
Re: powerpc ld.lld fix
Makes sense. This code is just the space reservation, the relocation generation or whatever fills them in is suppressed already, yes? Assuming so, r+ On Sat, Oct 31, 2020 at 2:46 PM Mark Kettenis wrote: > > Date: Sat, 10 Oct 2020 23:19:19 +0200 (CEST) > > From: Mark Kettenis > > > > On powerpc with the secure-plt ABI we need a .got section, even if the > > _GLOBAL_OFFSET_TABLE_ symbol isn't referenced. This is needed because > > the first three entries of the GOT are used by the dynamic linker. > > > > With this fix I can build executables of all flavours (including > > -static/-nopie). > > Turns out that adding these GOT entries when using "ld -r" is a bad > idea. Dif below fixes that. > > ok? > > > Index: gnu/llvm/lld/ELF/SyntheticSections.cpp > === > RCS file: /cvs/src/gnu/llvm/lld/ELF/SyntheticSections.cpp,v > retrieving revision 1.2 > diff -u -p -r1.2 SyntheticSections.cpp > --- gnu/llvm/lld/ELF/SyntheticSections.cpp 11 Oct 2020 13:10:13 > - 1.2 > +++ gnu/llvm/lld/ELF/SyntheticSections.cpp 31 Oct 2020 23:37:11 - > @@ -604,7 +604,7 @@ GotSection::GotSection() >// ElfSym::globalOffsetTable. >if (ElfSym::globalOffsetTable && !target->gotBaseSymInGotPlt) > numEntries += target->gotHeaderEntriesNum; > - else if (config->emachine == EM_PPC) > + else if (config->emachine == EM_PPC && !config->relocatable) > numEntries += target->gotHeaderEntriesNum; > } > > >
Re: Diff: Introductory Clause Comma Crap
On Sat, Oct 31, 2020 at 10:19 AM VARIK VALEFOR wrote: > This message contains even more grammatical fixes for the OpenBSD > manual pages. To ensure that the changes which are proposed in this > message can be justified, this message primarily fixes only a single > type of error: the absence of commas after introductory clauses. > As a procedural side-note, I would like to suggest that before going on a quest to make a change that touches so many files cross OpenBSD's code base, that it would be wise to send out a diff with just a couple examples and verify that the particular item of concern and the proposed fix is agreed upon before spending the time to search and edit many other pages. This is true not just for documentation changes but code changes, of course: doing lots of work before there's "buy-in" is risking your time. Philip Guenther
Re: [PATCH] ifconfig: keep track of allowed ips pointer correctly
On Tue, Oct 27, 2020 at 9:18 AM Jason A. Donenfeld wrote: ... > @@ -5721,16 +5720,18 @@ growwgdata(size_t by) > if (wg_aip != NULL) > wg_aip = (void *)wg_interface + aip_offset; > > - ret = (void *)wg_interface + wgdata.wgd_size - by; > - bzero(ret, by); > - > - return ret; > + bzero((void *)wg_interface + wgdata.wgd_size - by, by); > } > Total side note, but this caught my eye as relying on the gcc extension to do pointer arithmetic on "void *" pointers. At least historically we've avoided relying on that, which is easy in this case by just casting to "char *" instead. Philip Guenther
Re: [patch] const cclasses reference in ksh
On Mon, Oct 26, 2020 at 8:35 AM Matthew Martin wrote:
> Recently cclasses in lib/libc/gen/charclass.h was made const.[1]
> Mark the pointer used to walk the array in ksh const as well.
>
> 1: https://marc.info/?l=openbsd-cvs=160256416506433=2
>
Ugh, I totally missed that reach-around.
ok guenther@
> diff --git misc.c misc.c
> index 9e6e9db5e76..7226f74eccf 100644
> --- misc.c
> +++ misc.c
> @@ -713,7 +713,7 @@ do_gmatch(const unsigned char *s, const unsigned char
> *se,
> static int
> posix_cclass(const unsigned char *pattern, int test, const unsigned char
> **ep)
> {
> - struct cclass *cc;
> + const struct cclass *cc;
> const unsigned char *colon;
> size_t len;
> int rval = 0;
>
>
Re: Infinite loop in uvm protection mapping
On Mon, Oct 19, 2020 at 3:13 PM Tom Rollet wrote: > Hi, > > I'm starting to help in the development of the dt device. > > I'm stuck on permission handling of memory. I'm trying to allocate a > page in kernel with read/write protections, fill the allocated page > with data then change the permissions to read/exec. > > Snippet of my code: > > addr = uvm_km_alloc(kernel_map, PAGE_SIZE); > > [...] (memcpy data in allocated page) > > uvm_map_protect(kernel_map, addr, addr + PAGE_SIZE, PROT_READ > | PROT_EXEC, FALSE))) > This is same usage as seen in the 'sti' driver...which is on hppa only, so while it's presumably the correct usage of uvm_km_alloc() and uvm_map_protect(), I don't think uvm_map_protect() has been used on kernel-space on amd64 (or possibly all non-hppa archs) before in OpenBSD. Whee? It triggers the following error at boot time when executing > the uvm_map_protect function. > > uvm_fault(0x81fb2c90, 0x7ffec0008000, 0, 2) -> e kernel: page fault > trap, code=0 Stopped atpmap_write_protect+0x1f5: lock andq > $-0x3,0(%rdi) > > Trace: > > pmap_write_protect(82187b28,80002255b000,80002255c000, > 5,50e8b70481f4f622,fd81b6567e70) at pmap_write_protect+0x212 > uvm_map_protect(82129ae0,80002255b000,80002255c000 > ,5,0,82129ae0) at uvm_map_protect+0x501 > dt_alloc_kprobe(815560e0,80173900,e7ef01a2855152cc, > 82395c98,0,815560e0) at dt_alloc_kprobe+0x1ff > dt_prov_kprobe_init(2333e28db00d3edd,0,82121150,0,0, > 824d9008) at dt_prov_kprobe_init+0x1d9 > dtattach(1,821fb384,f,1,c2ee1c3f472154e,2dda28) at dtattach+0x5d > main(0,0,0,0,0,1) at main+0x419 > > The problem comes from the loop in pmap_write_protect > (sys/arch/amd64/amd64/pmap.c:2108) that is executed > infinity in my case. > > Entry of function pmap_write_protect: > sva: 80002250A000 > eva: 80002250B000 > > After &= PG_FRAME (line 2098-2099) > sva= F80002250A000 > eva= F80002250B000 > > loop: (ligne 2108) > > first iteration: > va = F80002250A000 > eva = F80002250B000 > blockend = 080012240 > ... > Does anyone have an idea how to fix this issue? So, blockend is clearly wrong for va and eva. I suspect the use of L2_FRAME here: blockend = (va & L2_FRAME) + NBPD_L2; is wrong here and it should be blockend = (va & VA_SIGN_NEG(L2_FRAME)) + NBPD_L2; or some equivalent expression to keep all the bits above the frame. Philip Guenther
Re: incorrect result from getppid for ptraced processes
On Fri, Sep 4, 2020 at 2:59 PM Mateusz Guzik wrote:
> On 9/5/20, Philip Guenther wrote:
> > On Fri, Sep 4, 2020 at 1:06 PM Mateusz Guzik wrote:
> >
> >> On 9/4/20, Vitaliy Makkoveev wrote:
> >> > On Fri, Sep 04, 2020 at 05:24:42PM +0200, Mateusz Guzik wrote:
> >> >> getppid blindly follows the parent pointer and reads the pid.
> >> >>
> >> >> The problem is that ptrace reparents the traced process, so in
> >> >> particular if you gdb -p $something, the target proc will start
> seeing
> >> >> gdb instead of its actual parent.
> >> >>
> >> >> There is a lot to say about the entire reparenting business or
> storing
> >> >> the original pid in ps_oppid (instead of some form of a reference to
> >> >> the process).
> >> >>
> >> >> However, I think the most feasible fix for now is the same thing
> >> >> FreeBSD did: *always* store the actual parent pid in ps_oppid. This
> >> >> means all repareting will keep updating it (most notably when
> >> >> abandoning children on exit), while ptrace will skip that part.
> >> >>
> >> >> Side effect of such a change be that getppid will stop requiring the
> >> >> kernel lock.
> >> >>
> >> >
> >> > Thanks for report. But we are in beta stage now so such modification
> is
> >> > impossible until next iteration.
> >> >
> >> > Since original parent identifier is stored as `ps_oppid' while process
> >> > is traced we just return it to userland for this case. This is the way
> >> > I
> >> > propose to fix this bug for now.
> >> >
> >> > Comments? OKs?
> >> >
> >> > Index: sys/kern/kern_prot.c
> >> > ===
> >> > RCS file: /cvs/src/sys/kern/kern_prot.c,v
> >> > retrieving revision 1.76
> >> > diff -u -p -r1.76 kern_prot.c
> >> > --- sys/kern/kern_prot.c 9 Jul 2019 12:23:25 - 1.76
> >> > +++ sys/kern/kern_prot.c 4 Sep 2020 21:12:15 -
> >> > @@ -84,7 +84,11 @@ int
> >> > sys_getppid(struct proc *p, void *v, register_t *retval)
> >> > {
> >> >
> >> > - *retval = p->p_p->ps_pptr->ps_pid;
> >> > + if (p->p_p->ps_flags & PS_TRACED)
> >> > + *retval = p->p_p->ps_oppid;
> >> > + else
> >> > + *retval = p->p_p->ps_pptr->ps_pid;
> >> > +
> >> > return (0);
> >> > }
> >>
> >> This is definitely a bare minimum fix, but it does the job.
> >>
> >
> > ptrace() has behaved like this for the life of OpenBSD and an indefinite
> > number of years previous in the BSD releases. What has happened that a
> > definitely incomplete fix is needed Right Now?
>
> I don't see how this reads as a demand this is fixed Right Now.
>
I didn't call it a demand, but the point stands: what has changed?
I don't see how the fix is incomplete either. It can be done better
> with more effort, but AFAICS the above results in correct behavior.
>
There are at least 2 other uses of ps_pptr->ps_pid that should also change,
unless you like coredumps and ps disagreeing with getppid(), and someone
needs to think how it affects doas.
Philip Guenther
Re: incorrect result from getppid for ptraced processes
On Fri, Sep 4, 2020 at 1:06 PM Mateusz Guzik wrote:
> On 9/4/20, Vitaliy Makkoveev wrote:
> > On Fri, Sep 04, 2020 at 05:24:42PM +0200, Mateusz Guzik wrote:
> >> getppid blindly follows the parent pointer and reads the pid.
> >>
> >> The problem is that ptrace reparents the traced process, so in
> >> particular if you gdb -p $something, the target proc will start seeing
> >> gdb instead of its actual parent.
> >>
> >> There is a lot to say about the entire reparenting business or storing
> >> the original pid in ps_oppid (instead of some form of a reference to
> >> the process).
> >>
> >> However, I think the most feasible fix for now is the same thing
> >> FreeBSD did: *always* store the actual parent pid in ps_oppid. This
> >> means all repareting will keep updating it (most notably when
> >> abandoning children on exit), while ptrace will skip that part.
> >>
> >> Side effect of such a change be that getppid will stop requiring the
> >> kernel lock.
> >>
> >
> > Thanks for report. But we are in beta stage now so such modification is
> > impossible until next iteration.
> >
> > Since original parent identifier is stored as `ps_oppid' while process
> > is traced we just return it to userland for this case. This is the way I
> > propose to fix this bug for now.
> >
> > Comments? OKs?
> >
> > Index: sys/kern/kern_prot.c
> > ===
> > RCS file: /cvs/src/sys/kern/kern_prot.c,v
> > retrieving revision 1.76
> > diff -u -p -r1.76 kern_prot.c
> > --- sys/kern/kern_prot.c 9 Jul 2019 12:23:25 - 1.76
> > +++ sys/kern/kern_prot.c 4 Sep 2020 21:12:15 -
> > @@ -84,7 +84,11 @@ int
> > sys_getppid(struct proc *p, void *v, register_t *retval)
> > {
> >
> > - *retval = p->p_p->ps_pptr->ps_pid;
> > + if (p->p_p->ps_flags & PS_TRACED)
> > + *retval = p->p_p->ps_oppid;
> > + else
> > + *retval = p->p_p->ps_pptr->ps_pid;
> > +
> > return (0);
> > }
>
> This is definitely a bare minimum fix, but it does the job.
>
ptrace() has behaved like this for the life of OpenBSD and an indefinite
number of years previous in the BSD releases. What has happened that a
definitely incomplete fix is needed Right Now?
Philip Guenther
Re: timekeep: fixing large skews on amd64 with RDTSCP
On Thu, Jul 16, 2020 at 4:55 PM Scott Cheloha wrote: > > On Jul 16, 2020, at 19:36, Theo de Raadt wrote: > > > >> Note the third sentence. > >> > >> Given that, I reason that a serializing instruction before *and* after > >> the RDTSC should freeze it in place. > > > > I haven't seen anyone read it that way. > > They say that instructions after RDTSC can run before it because > it isn't a serializing instruction. > > Do we want that? > > And then, consider this bit of programming advice. Also from the > ISA reference (Vol. 2B 4-547): > > > If software requires RDTSC to be executed prior to execution of any > > subsequent instruction (including any memory accesses), it can > > execute the sequence LFENCE immediately after RDTSC. > > What other reading is possible given this documentation? > > What is your interpretation? Am I missing something? > If you're trying to time a sequence of instructions via rdtsc, then you have to put an lfence after the first rdtsc (so that the sequence being timed doesn't get lifted to before it completes) and an lfence before the second rdtsc (so that the sequence is actually complete before the second one). In this case, is it a problem if instructions after the rdtsc that are not dependent on its result may be started before it's actually complete? If not, then there's no reason to bracket that side. Philip Guenther
Re: disable libc sys wrappers?
On Wed, Jul 8, 2020 at 8:06 AM Theo de Raadt wrote: > Mark Kettenis wrote: > > > > From: "Theo de Raadt" > > > Date: Wed, 08 Jul 2020 09:42:41 -0600 > > > > > > I think we need something like this. > > > > > > Documenting it will be a challenge. > > > > > > I really don't like the name as is too generic, when the control is > only > > > for a narrow set of "current time" system calls. > > > > I'm not sure we should be using getenv() in this early initialization > > function though. > > Ah, you worry about the static "#ifndef PIC / early_static_init" versus > "PIC ld.so" environ setup, and this very early getenv() call might not be > looking at the environ global. > It's late enough in the process (after a possible call to early_static_init(), and definitely after any fixup by ld.so) that it should work Just Fine. I would flip the test to check the environment before running issetugid(2) because the syscall is more expensive and it's nice not to clutter the kdump output. ;-) Philip Guenther
Re: SSE in kernel?
On Mon, Jun 22, 2020 at 9:12 PM wrote: > Are SSE instructions allowed in the AMD64 kernel? Is #ifdef __SSE__ > a sufficient guard? > > I have a rasops32 putchar with SSE that is 2x faster. > As Bryan and Patrick noted: it's possible, but there are restrictions and costs. The main restriction is that the code must not permit a context-switch between the fpu_kernel_enter() and fpu_kernel_exit() calls. No taking an rwlock or calling any of the sleep functions, for example. If you're using more than the minimal level of SSE which is already required by the kernel (for lfence, etc) then you should also check whether the necessary extension bits are present in curcpu()->ci_feature_* and fall back to the current code if not present. The cost is that if the thread doing this isn't a system thread, then the first fpu_kernel_enter() call after the userspace->kernel transition has to save and reset the FPU registers (XSAVEOPT + XRSTOR on newish CPUs). Every fpu_kernel_exit(), regardless of thread type, resets them again (XRSTOR). If the restriction isn't a problem and the cost of those is worth the gain, then sure, go for it. We already do it for AES stuff in the kernel, for example. c.f. /usr/src/sys/arch/amd64/amd64/aesni.c Philip Guenther
Re: EV_SET(2) shadows variable
On Sat, 4 Apr 2020, Theo de Raadt wrote:
> Philip Guenther wrote:
>
> > On Fri, 3 Apr 2020, Martin Pieuchot wrote:
> > > Thanks, here it is, ok?
> >
> > ok guenther@
>
> Should we do the same to all other macros, just in case?
Checking /usr/include/{,sys/}*.h, the diff below fixes the only ones I
found to be potential problems
/usr/include/net* and some others have not-completely-safe macros, like
IP6_EXTHDR_GET()
Index: include/bitstring.h
===
RCS file: /data/src/openbsd/src/include/bitstring.h,v
retrieving revision 1.5
diff -u -p -r1.5 bitstring.h
--- include/bitstring.h 2 Jun 2003 19:34:12 - 1.5
+++ include/bitstring.h 6 Apr 2020 00:37:52 -
@@ -83,46 +83,46 @@ typedef unsigned char bitstr_t;
/* clear bits start ... stop in bitstring */
#definebit_nclear(name, start, stop) do { \
- register bitstr_t *_name = name; \
- register int _start = start, _stop = stop; \
- while (_start <= _stop) { \
- bit_clear(_name, _start); \
- _start++; \
+ register bitstr_t *__name = (name); \
+ register int ___start = (start), __stop = (stop); \
+ while (__start <= __stop) { \
+ bit_clear(__name, __start); \
+ __start++; \
} \
} while(0)
/* set bits start ... stop in bitstring */
#definebit_nset(name, start, stop) do { \
- register bitstr_t *_name = name; \
- register int _start = start, _stop = stop; \
- while (_start <= _stop) { \
- bit_set(_name, _start); \
- _start++; \
+ register bitstr_t *__name = (name); \
+ register int __start = (start), __stop = (stop); \
+ while (__start <= __stop) { \
+ bit_set(__name, __start); \
+ __start++; \
} \
} while(0)
/* find first bit clear in name */
#definebit_ffc(name, nbits, value) do { \
- register bitstr_t *_name = name; \
- register int _bit, _nbits = nbits, _value = -1; \
- for (_bit = 0; _bit < _nbits; ++_bit) \
- if (!bit_test(_name, _bit)) { \
- _value = _bit; \
+ register bitstr_t *__name = (name); \
+ register int __bit, __nbits = (nbits), __value = -1; \
+ for (__bit = 0; __bit < __nbits; ++__bit) \
+ if (!bit_test(__name, __bit)) { \
+ __value = __bit; \
break; \
} \
- *(value) = _value; \
+ *(value) = __value; \
} while(0)
/* find first bit set in name */
#definebit_ffs(name, nbits, value) do { \
- register bitstr_t *_name = name; \
- register int _bit, _nbits = nbits, _value = -1; \
- for (_bit = 0; _bit < _nbits; ++_bit) \
- if (bit_test(_name, _bit)) { \
- _value = _bit; \
+ register bitstr_t *__name = (name); \
+ register int __bit, __nbits = (nbits), __value = -1; \
+ for (__bit = 0; __bit < __nbits; ++__bit) \
+ if (bit_test(__name, __bit)) { \
+ __value = __bit; \
break; \
} \
- *(value) = _value; \
+ *(value) = __value; \
} while(0)
#endif /* !_BITSTRING_H_ */
Index: sys/sys/disklabel.h
===
RCS file: /data/src/openbsd/src/sys/sys/disklabel.h,v
retrieving revision 1.75
diff -u -p -r1.75 disklabel.h
--- sys/sys/disklabel.h 24 Oct 2017 09:36:13 - 1.75
+++ sys/sys/disklabel.h 6 Apr 2020 00:52:08 -
@@ -156,37 +156,37 @@ struct__partitionv0 { /* old (v0) part
#define DL_GETPSIZE(p) (((u_int64_t)(p)->p_sizeh << 32) + (p)->p_size)
#define DL_SETPSIZE(p, n) do { \
- u_int64_t x = (n); \
- (p)->p_sizeh = x >> 32; \
- (p)->p_size = x; \
+ u_int64_t __x = (n); \
+ (p)->p_sizeh = __x >> 32; \
+ (p)->p_size = __x; \
} while (0)
#define DL_GETPOFFSET(p) (((u_int64_t)(p)->p_offseth << 32) +
(p)->p_offset)
#define DL_SETPOFFSET(p, n)do { \
- u_int64_t x = (n); \
- (p)->p_offseth = x >> 32; \
- (p)->p_offset = x; \
+ u_int64_t __x = (n); \
+ (p)->p_offseth = __x >> 32; \
+
Re: split futex into three
On Sun, 5 Apr 2020, Stuart Henderson wrote: > On 2020/04/05 10:28, Martin Pieuchot wrote: > > Another way to proceed would be to do a port grep for futex and see what > > the ecosystem is using. > > Sorry it's not filtered, but : > > https://junkpile.org/grep.futex.gz Sure looks like the only occurence of futex() used with FUTEX_REQUEUE (== 3) is the linux kernel test program. Everything else, including rust, is using FUTEX_CMP_REQUEUE or one of the PI operations (FUTEX_WAIT_REQUEUE_PI, FUTEX_CMP_REQUEUE_PI). Philip
Re: kdump futex fix
On Sat, 4 Apr 2020, Martin Pieuchot wrote: > On 03/04/20(Fri) 19:26, Philip Guenther wrote: > > On Fri, 3 Apr 2020, Martin Pieuchot wrote: > > > Depending on the operation requested futex(2) might return the number of > > > woken threads or an error. That means the following... > > > > > > mpv CALL > > > futex(0xa58935899b0,0x82,1,0,0) > > > mpv RET futex -1 errno 1 Operation not permitted > > > > > > ...is not an error but it indicates that 1 thread has been awoken. > > > > > > Diff below would have saved me quite some time looking in the wrong > > > direction. > > > > > > ok? > > > > Isn't that just a complicated way to get the same effect as deleting the > > "case SYS_futex:" line shortly above there? > > Thanks! Updated below :) ok guenther@
Re: split futex into three
On Sat, 4 Apr 2020, Paul Irofti wrote:
> Here is a proper diff (both sys and libc into one).
Okay, bunch o' comments and thoughts of varying strength of opinion.
> diff --git lib/libc/Symbols.list lib/libc/Symbols.list
> index f9aa62ab6e8..4fa37a835aa 100644
> --- lib/libc/Symbols.list
> +++ lib/libc/Symbols.list
> @@ -86,7 +86,10 @@ _thread_sys_fstatat
> _thread_sys_fstatfs
> _thread_sys_fsync
> _thread_sys_ftruncate
> -_thread_sys_futex
> +_thread_sys_ofutex
The ofutex syscall should not be built (or exported) by libc. The kernel
needs to continue to provide it for the normal period to support older
libc's where futex(2) is a direct syscall,but no application does, would,
or should invoke ofutex() under that name.
> +_thread_sys_futex_wait
> +_thread_sys_futex_wake
> +_thread_sys_futex_requeue
glibc has internal inline functions futex_wait() and futex_wake() and
there has been at least discussion about exporting some version of them.
If our signatures matched the last-best-proposal over there (which was
dropped, mind you) then I would be tempted to use those names. If not,
then maybe go with _futex_wait() and _futex_wake()?
FUTEX_REQUEUE is the old bad one, with no val3 argument that's checked
before the operation. Our libc/libpthread don't actually use them, and in
the Linux world glibc switched completely to FUTEX_CMP_REQUEUE. Perhaps
we should drop support for FUTEX_REQUEUE (major bump, yah) and add
_futex_cmp_requeue(2) when we need it?
> --- lib/libc/gen/sigwait.c
> +++ lib/libc/gen/sigwait.c
This is unrelated to futex stuff. :)
> --- lib/libc/hidden/sys/futex.h
> +++ lib/libc/hidden/sys/futex.h
> @@ -20,6 +20,8 @@
>
> #include_next
>
> -PROTO_NORMAL(futex);
You need to keep this as long as futex() is used internally. Once futex()
is no longer used in libc then this should change to
"PROTO_DEPRECATED(futex);" and the "DEF_STRONG(futex);" line should be
deleted.
(If you have DEF_STRONG(foo) or DEF_WEAK(foo), then you must have
PROTO_NORMAL(foo). c.f libc/include/README II.B.2)
> --- lib/libc/thread/synch.h
> +++ lib/libc/thread/synch.h
> @@ -19,10 +19,33 @@
> #include
> #include
>
> +static inline int
> +_futex(volatile uint32_t *p, int op, int val, const struct timespec
> *timeout, uint32_t *g)
> +{
> + int flags = 0;
> +
> + if (op & FUTEX_PRIVATE_FLAG)
> + flags |= FT_PRIVATE;
> +
> + switch (op) {
> + case FUTEX_WAIT:
> + case FUTEX_WAIT_PRIVATE:
> + return futex_wait(p, val, timeout, flags);
> + case FUTEX_WAKE:
> + case FUTEX_WAKE_PRIVATE:
> + return futex_wake(p, val, flags);
> + case FUTEX_REQUEUE:
> + case FUTEX_REQUEUE_PRIVATE:
> + return futex_requeue(p, val, g, timeout, flags);
> + }
> +
> + return ENOSYS;
> +}
I would put this logic directly into futex()'s definition, and then...
> static inline int
> _wake(volatile uint32_t *p, int n)
> {
> - return futex(p, FUTEX_WAKE_PRIVATE, n, NULL, NULL);
> + return _futex(p, FUTEX_WAKE_PRIVATE, n, NULL, NULL);
> }
have this just be
return futex_wake(p, n, FUTEX_WAKE_PRIVATE);
and similar for _twait() below.
(I would be inclined to use FUTEX_WAKE_PRIVATE as the flag for the new
syscalls instead of exposing FT_PRIVATE into the userspace API.)
> --- sys/kern/sys_futex.c
> +++ sys/kern/sys_futex.c
> @@ -83,9 +83,74 @@ futex_init(void)
> }
>
> int
> -sys_futex(struct proc *p, void *v, register_t *retval)
> +sys_futex_wait(struct proc *p, void *v, register_t *retval)
> {
> - struct sys_futex_args /* {
> + struct sys_futex_wait_args /* {
> + syscallarg(uint32_t *) f;
> + syscallarg(inr) val;
> + syscallarg(const struct timespec *) timeout;
> + syscallarg(int) flags;
> + } */ *uap = v;
> + uint32_t *uaddr = SCARG(uap, f);
> + uint32_t val = SCARG(uap, val);
> + const struct timespec *timeout = SCARG(uap, timeout);
> + int flags = SCARG(uap, flags);
> +
> + KERNEL_LOCK();
> + rw_enter_write();
> + *retval = futex_wait(uaddr, val, timeout, flags);
> + rw_exit_write();
> + KERNEL_UNLOCK();
> +
> + return 0;
> +}
> +
> +int
> +sys_futex_wake(struct proc *p, void *v, register_t *retval)
> +{
> + struct sys_futex_wake_args /* {
> + syscallarg(uint32_t *) f;
> + syscallarg(int) val;
> + syscallarg(int) flags;
> + } */ *uap = v;
> + uint32_t *uaddr = SCARG(uap, f);
> + uint32_t val = SCARG(uap, val);
> + int flags = SCARG(uap, flags);
> +
> + rw_enter_write();
> + *retval = futex_wake(uaddr, val, flags);
> + rw_exit_write();
> +
> + return 0;
> +}
> +
> +int
> +sys_futex_requeue(struct proc *p, void *v, register_t *retval)
> +{
> + struct sys_futex_requeue_args /* {
> + syscallarg(uint32_t *) f;
> + syscallarg(int) val;
> + syscallarg(uint32_t *) g;
> +
Re: kdump futex fix
On Fri, 3 Apr 2020, Martin Pieuchot wrote: > Depending on the operation requested futex(2) might return the number of > woken threads or an error. That means the following... > > mpv CALL futex(0xa58935899b0,0x82,1,0,0) > mpv RET futex -1 errno 1 Operation not permitted > > ...is not an error but it indicates that 1 thread has been awoken. > > Diff below would have saved me quite some time looking in the wrong > direction. > > ok? Isn't that just a complicated way to get the same effect as deleting the "case SYS_futex:" line shortly above there? The real problem is that futex(2) is actually 3 different syscalls wrapped into one. It was split into three then kdump could properly report futex_wake(2) and futex_requeue(2) as returning a count, while futex_wait(2) returns an errno. The existing 'switch' in sys_futex() would just move to userspace's futex(3), provided for linux compat. (And our syscalls would have proper prototypes and futex_wait() could take a clockid_t, and)
Re: EV_SET(2) shadows variable
On Fri, 3 Apr 2020, Martin Pieuchot wrote: > Thanks, here it is, ok? ok guenther@
Re: EV_SET(2) shadows variable
On Tue, Mar 31, 2020 at 11:24 PM Martin Pieuchot wrote:
> The current form of EV_SET(2) declares a `kevp' variable. This can
> cause to subtle bugs hard to discover if one uses a variable of the
> same to retrieve events.
>
> Diff below prefixes the locally declared variable by an underscore,
> like it it done in FD_ZERO(), which should be good enough to prevent
> surprises.
>
> Is it the right way to correct such issue? How many underscores are
> enough? Can the standards gurus tell me?
>
tl;dr: this should use a variable that starts with either two underbars
(__kevp) or an underbar and a capital (_Kevp). The same is true of
FD_ZERO().
The namespace of identifiers that begin with a single underbar is only
"reserved for use as identifiers with file scope in both the ordinary and
tag name spaces". That means it's perfectly legal for an application to
declare a *local* variable with such a name. So, this should be legal:
#include
#include
#include
#include
int main(void)
{
struct fd_set _p;
struct kevent _kevp;
FD_ZERO(&_p):
EV_SET(&_kevp, 0, 0, 0, 0, 0, 0);
return 0;
}
...but it currently blows up on the FD_ZERO() and would blow up in the
EV_SET() with your proposed diff.
Philip Guenther
