[2 WEEKS LEFT REMINDER] Call for 2023Q3 status reports

[Lorenzo Salvadore]

Dear FreeBSD Community,

The deadline for the next FreeBSD Status Report update is
September, 30th 2023 for work done since the last round of quarterly reports:
July 2023 - September 2023.
I would like to remind you that reports are published on a quarterly
basis and are usually collected during the last month of each quarter,
You are also welcome to submit them even earlier if you want, and the
earlier you submit them, the more time we have for reviewing.

Status report submissions do not need to be very long. They may be
about anything happening in the FreeBSD project and community, and
they provide a great way to inform FreeBSD users and developers about
work that is underway or has been completed. Report submissions are
not limited to committers; anyone doing anything interesting and
FreeBSD related can -- and should -- write one!

The following methods are available to submit your reports:

* submit a review on Phabricator and add the group "status" to the
  reviewers list. You should put your reports in the directory
  doc/website/content/en/status/report-2023-07-2023-09/
  (create it if it is missing);

* submit a pull request at .
  You should put your reports in the directory
  doc/website/content/en/status/report-2023-07-2023-09/
  (create it if it is missing);

* send an email to status-submissi...@freebsd.org including your report.

An AsciiDoc template is available at
.

We look forward to seeing your 2023Q3 reports!

Thanks,

Lorenzo Salvadore (on behalf of status@)

A lock order reversal that I've not seen before (zfs and tmpfs during poudriere bulk using USE_TMPFS=all)

[Mark Millard]

I've never figured out how to tell important lock order reversal
notices from unimportant ones. So I mostly report only unfamiliar
ones. (But I normally do not do poudriere bulk builds with a
debug kernel in use.)

During a poudriere bulk that is using USE_TMPFS=all it reported:

lock order reversal:
 1st 0xa0027b7e83f0 zfs (zfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:2240
 2nd 0xa0023db44070 tmpfs (tmpfs, lockmgr) @ 
/usr/src/sys/kern/vfs_subr.c:3886
lock order tmpfs -> zfs established at:
#0 0x004d7824 at witness_checkorder+0x304
#1 0x00435bfc at lockmgr_xlock+0x50
#2 0x00015d0ce814 at null_lock+0xb0
#3 0x0056cdf0 at _vn_lock+0x54
#4 0x00557170 at vflush+0x12c
#5 0x00015d0cd6b0 at nullfs_unmount+0x40
#6 0x0054c318 at dounmount+0x714
#7 0x0054bb94 at kern_unmount+0x298
#8 0x007fe6ac at do_el0_sync+0x520
#9 0x007da110 at handle_el0_sync+0x44
lock order zfs -> tmpfs attempted at:
#0 0x004d7fb8 at witness_checkorder+0xa98
#1 0x00434140 at lockmgr_lock_flags+0x1ec
#2 0x0056cdf0 at _vn_lock+0x54
#3 0x00557170 at vflush+0x12c
#4 0x003964d0 at tmpfs_unmount+0x60
#5 0x0054c318 at dounmount+0x714
#6 0x0054bb94 at kern_unmount+0x298
#7 0x007fe6ac at do_el0_sync+0x520
#8 0x007da110 at handle_el0_sync+0x44

It happens to be on aarch64, in case that matters for some
odd reason.

===
Mark Millard
marklmi at yahoo.com

Fwd: git: 8d49fd7331bc - main - pf: remove DIOCGETRULE and DIOCGETSTATUS : net/py-libdnet and net/scapy now broken, kyua test suite damaged

[Mark Millard]

[I've cc'd a couple of folks that have dealt with fixing
breakage in the past.]

From: Kristof Provost 
Subject: Re: git: 8d49fd7331bc - main - pf: remove DIOCGETRULE and 
DIOCGETSTATUS : net/py-libdnet and net/scapy now broken, kyua test suite damaged
Date: September 14, 2023 at 02:02:38 PDT
To: Mark Millard 
Cc: Current FreeBSD 
> 
> Hi Mark,
> 
> On 14 Sep 2023, at 7:37, Mark Millard wrote:
>> This change leads the port net/py-libdnet to be broken:
>> 
>> --- fw-pf.lo ---
>> fw-pf.c:212:22: error: use of undeclared identifier 'DIOCGETRULE'
>> if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
>> ^
>> fw-pf.c:252:22: error: use of undeclared identifier 'DIOCGETRULE'
>> if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
>> ^
>> --- intf.lo ---
>> for (cnt = 0; !matched && cnt < (int) entry->intf_alias_num; cnt++) {
>> ^
>> intf.c:571:2: note: previous statement is here
>> if (entry->intf_addr.addr_type == ADDR_TYPE_IP &&
>> ^
>> --- fw-pf.lo ---
>> fw-pf.c:296:28: error: use of undeclared identifier 'DIOCGETRULE'
>> if ((ret = ioctl(fw->fd, DIOCGETRULE, )) < 0)
>> ^
>> 3 errors generated.
>> 
>> That leads to:
>> 
>> [00:00:41] [29] [00:00:26] Finished net/py-libdnet@py39 | 
>> py39-libdnet-1.13_4: Failed: build
>> [00:00:42] [29] [00:00:27] Skipping net/scapy@py39 | py39-scapy-2.5.0_1: 
>> Dependent port net/py-libdnet@py39 | py39-libdnet-1.13_4 failed
>> 
> 
> The commit removed those ioctls because they’ve been superseded by newer 
> (nvlist-based) versions.
> Ports are strongly advised to use libpfctl rather than trying to deal with 
> nvlists themselves.
> 
> See https://lists.freebsd.org/archives/freebsd-pf/2023-April/000345.html for 
> an example of what the ports will have to do. It’s generally a trivial change.
> 
> Best regards,
> Kristof

===
Mark Millard
marklmi at yahoo.com

Re: Continually count the number of open files

[Mateusz Guzik]

On 9/13/23, David Chisnall  wrote:
> On 12 Sep 2023, at 17:19, Bakul Shah  wrote:
>>
>> FreeBSD
>> should add inotify.
>
> inotify is also probably not the right thing.  If someone is interested in
> adding this, Apple’s fsevents API is a better inspiration.  It is carefully
> designed to ensure that the things monitoring for events can’t ever block
> filesystem operations from making progress.

I'm not sure what you mean here specifically and I don't see anything
careful about what they did.

>From userspace POV the API is allowed to drop events, which makes life
easy on this front and is probably the right call.

The implementation is utterly horrid. For one, the non-blocking aspect
starts with the obvious equivalent of uma_zalloc(..., M_NOWAIT) and
bailing if it fails, except if you read past that to actual
registration it can perform an alloc which can block indefinitely
while holding on to some vnodes:
// if we haven't gotten the path yet, get it.
if (pathbuff == NULL) {
pathbuff = get_pathbuff();
pathbuff_len = MAXPATHLEN;

where get_pathbuf is:
return zalloc(ZV_NAMEI);

So the notification routine can block indefinitely in a low-memory
condition. I tried to figure out if this is ever called without other
vnodes write-locked (as opposed to "just" refed), but their code is
such a mess that my level of curiosity was dwarfed by difficulty of
getting a definitive answer.

Other than that it is terribly inefficient and artificially limited to
8 processes which can do anything.

That is to say it is unfit for anything but laptop-scale usage.

Perhaps you meant it does not block if the watchers decide to not
process any events, but that's almost inherently true if one allows
for lossy notifications.

> I think there’s a nice design
> possible with a bloom filter in the kernel of events that ensures that
> monitors may get spurious events but don’t miss out on anything.
>
[snip]
>  I think the right kernel API would walk the directory and add the vnodes to
> a bloom filter and trigger a notification on a match in the filter.  You’d
> then have occasional spurious notifications but you’d have something that
> could be monitored via kqueue and could be made to not block anything else
> in the kernel.
>

I don't see how this can work.

A directory can have more inodes than you can have vnodes at any
point. So if you add vnodes to a list as you go, they may fall off of
so that you can continue adding other entries.

But perhaps you mean you could store the inode number as opposed to
holding to the vnode? Even then, the number of entries to scan to make
it happen is so big that it is going to be impractical on anything but
laptop-scale.

What can be fast is checking if the parent dir wants notifications,
but this ignores changes to hardlinks. Except *currently* the VFS
layer does not reliably track who the parent is (and in fact it can
fail to spot one).

The VFS layer contains a lot of cruft and design decisions which at
least today are questionable at best, but fixable. A big chunk of this
concerns name caching, which currently is entirely optional. Should
someone want to propose an API for file notification changes, they
need to state something which if implemented does not result in
unfixable drag on the layer, even if initial implementation would be
suboptimal. Handling arbitrary hardlinks looks like a drag to me, but
I'm happy to review an implementation which avoids being a problem.

That is to say, a laptop-scale API can probably be implemented as is,
but solution which can provide reliable events (not to be confused
with reliably notifying about all events) would require numerous
changes.

-- 
Mateusz Guzik 

Re: git: 8d49fd7331bc - main - pf: remove DIOCGETRULE and DIOCGETSTATUS : net/py-libdnet and net/scapy now broken, kyua test suite damaged

[Kristof Provost]

Hi Mark,

On 14 Sep 2023, at 7:37, Mark Millard wrote:
> This change leads the port net/py-libdnet to be broken:
>
> --- fw-pf.lo ---
> fw-pf.c:212:22: error: use of undeclared identifier 'DIOCGETRULE'
> if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
> ^
> fw-pf.c:252:22: error: use of undeclared identifier 'DIOCGETRULE'
> if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
> ^
> --- intf.lo ---
> for (cnt = 0; !matched && cnt < (int) entry->intf_alias_num; cnt++) {
> ^
> intf.c:571:2: note: previous statement is here
> if (entry->intf_addr.addr_type == ADDR_TYPE_IP &&
> ^
> --- fw-pf.lo ---
> fw-pf.c:296:28: error: use of undeclared identifier 'DIOCGETRULE'
> if ((ret = ioctl(fw->fd, DIOCGETRULE, )) < 0)
> ^
> 3 errors generated.
>
> That leads to:
>
> [00:00:41] [29] [00:00:26] Finished net/py-libdnet@py39 | 
> py39-libdnet-1.13_4: Failed: build
> [00:00:42] [29] [00:00:27] Skipping net/scapy@py39 | py39-scapy-2.5.0_1: 
> Dependent port net/py-libdnet@py39 | py39-libdnet-1.13_4 failed
>

The commit removed those ioctls because they’ve been superseded by newer 
(nvlist-based) versions.
Ports are strongly advised to use libpfctl rather than trying to deal with 
nvlists themselves.

See https://lists.freebsd.org/archives/freebsd-pf/2023-April/000345.html for an 
example of what the ports will have to do. It’s generally a trivial change.

Best regards,
Kristof

Re: git: 8d49fd7331bc - main - pf: remove DIOCGETRULE and DIOCGETSTATUS : net/py-libdnet and net/scapy now broken, kyua test suite damaged

[Mark Millard]

This change leads the port net/py-libdnet to be broken:

--- fw-pf.lo ---
fw-pf.c:212:22: error: use of undeclared identifier 'DIOCGETRULE'
if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
^
fw-pf.c:252:22: error: use of undeclared identifier 'DIOCGETRULE'
if (ioctl(fw->fd, DIOCGETRULE, ) == 0 &&
^
--- intf.lo ---
for (cnt = 0; !matched && cnt < (int) entry->intf_alias_num; cnt++) {
^
intf.c:571:2: note: previous statement is here
if (entry->intf_addr.addr_type == ADDR_TYPE_IP &&
^
--- fw-pf.lo ---
fw-pf.c:296:28: error: use of undeclared identifier 'DIOCGETRULE'
if ((ret = ioctl(fw->fd, DIOCGETRULE, )) < 0)
^
3 errors generated.

That leads to:

[00:00:41] [29] [00:00:26] Finished net/py-libdnet@py39 | py39-libdnet-1.13_4: 
Failed: build
[00:00:42] [29] [00:00:27] Skipping net/scapy@py39 | py39-scapy-2.5.0_1: 
Dependent port net/py-libdnet@py39 | py39-libdnet-1.13_4 failed

net/scapy is used by parts of the kyua testsuite (when installed,
anyway). So the kyua testsuite is now has damaged functionality
on main [so: 15].

===
Mark Millard
marklmi at yahoo.com