Re: OpenZFS imports, status update

[Mark Millard via freebsd-current]

tech-lists  wrote on
Date: Wed, 9 Jun 2021 02:13:07 +0100 :

> root_at_desktop:/usr/src# git remote prune freebsd
> fatal: 'freebsd' does not appear to be a git repository
> fatal: Could not read from remote repository.
> 
> Please make sure you have the correct access rights
> and the repository exists.

Just a note for potential future reference,
first showing showing an example of finding
what name(s) identify remotes:

# git remote show
freebsd

Then there is checking what remote branches are known
for one of those remotes (showing an example here):

# git remote show freebsd
* remote freebsd
  Fetch URL: https://git.FreeBSD.org/src.git
  Push  URL: https://git.FreeBSD.org/src.git
  HEAD branch: main
  Remote branches:
main tracked
refs/notes/commits   tracked
. . .
vendor/zstd  tracked
  Local branches configured for 'git pull':
mainmerges with remote main
releng/13.0 merges with remote releng/13.0
stable/13   merges with remote stable/13
  Local refs configured for 'git push':
mainpushes to main(local out of date)
releng/13.0 pushes to releng/13.0 (local out of date)
stable/13   pushes to stable/13   (local out of date)

There can be things in that output such as:

. . .
refs/remotes/freebsd/vendor/openzfs stale (use 'git remote prune' to remove)
. . .
vendor/openzfs/legacy   new (next fetch will store in 
remotes/freebsd)
vendor/openzfs/master   new (next fetch will store in 
remotes/freebsd)
vendor/openzfs/zfs-2.1-release  new (next fetch will store in 
remotes/freebsd)
. . .



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

OpenZFS Encryption: Docs, and re Metadata Leaks

[grarpamp]

On 4/17/20, Ryan Moeller  wrote:
>
>> On Apr 17, 2020, at 4:56 PM, Pete Wright  wrote:
>>
>> On 4/17/20 11:35 AM, Ryan Moeller wrote:
>>> OpenZFS brings many exciting features to FreeBSD, including:
>>>  * native encryption
>> Is there a good doc reference on available for using this?  I believe this
>> is zfs filesystem level encryption and not a replacement for our existing
>> full-disk-encryption scheme that currently works?
>
> I’m not aware of a good current doc for this. If anyone finds/writes
> something, please post it!
> There are some old resources you can find with a quick search that do a
> pretty good job of covering the basic ideas, but I think the exact syntax of
> commands may be slightly changed in the final implementation.
>
> The encryption is performed at a filesystem level (per-dataset).


You could find some initial doc and video about zfs encryption
on openzfs.org and youtube, and in some commit logs.
Therein was mentioned...

People are needed to volunteer to expand documentation on the
zfs crypto subject further in some document committed to openzfs
repo since users and orgs will want to know it when considering use.
Volunteers are also sought by openzfs to review the crypto itself.

Maybe there was already some central place made with further
current documentation about the zfs encryption topics since then?

https://www.youtube.com/watch?v=frnLiXclAMo openzfs encryption
https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view
https://www.youtube.com/watch?v=kFuo5bDj8C0 openzfs encryption cloud
https://drive.google.com/file/d/14uIZmJ48AfaQU4q69tED6MJf-RJhgwQR/view

It's dataset level, so GELI or GBDE etc are needed for full coverage,
perhaps even those two may not have yet received much or formal
review either, so there is always good volunteer opportunity there
to start with review of a potentially simpler cryptosystem like those.

zfs list, dataset snapshot names properties etc not covered.

zfs history not covered, many sensitive strings will end up in
there, including cutpaste password typos into commandline,
usernames, timestamps, etc...
and no tool exist to scrub overwrite history extents with random data,
and no option exists to turn keeping of
'user initiated events' or 'long format' off,
and ultimately no option exists to tell zpool create to
disable history keeping from the very start entirely.
So maybe users have to zero disks and pools along
with it just to scrub that.

zfs also exposes these variety of path and device names,
timestamps, etc in cleartext on disk structures in various
places, including configuration cachefile...
Some of those could could be NULLed or dummied
with new zpool create options for more security
restricted use cases.

There are other meta things and tools left exposed
such as potentially any plaintext meta in send/recv.

Another big metadata leak for environments and users that
ship, sell, embed, clone, distribute, fileshare, and backup,
their raw disks pools and usbs around to untrusted third parties...
is that zfs also puts hostnames and UUID type of unique
static meta and identifying things in cleartext on disk.
zfs thus needs options to allow users to set and use a NULL,
or generic dummy default, or random string, or chosen,
"hostname" for those from the very first zpool create command.

Most applications users use, including zfs, can today
consider ways in which metadata leaks could be removed
entirely, or at least optioned out for use under
high security restricted environments modes.
That could even involve considering trading off some
extra features not actually required for a basic mode
of functionality of the app.



(cc's for fyi inclusion about leaks, and as lists still haven't
been configured to support discreet bcc for that purpose,
which would also maintain nice headers for thread following.
Gmail breaks threads. zfsonlinux topicbox peole can't subscribe
without javabloatbroken website, so someone could forward
this there. Drop non-relevant cc's from further replies.
Parent thread from freebsd current and stable lists was
Subject: OpenZFS port updated)

Re: OpenZFS port updated

[grarpamp]

On 4/17/20, Ryan Moeller  wrote:
>
>> On Apr 17, 2020, at 4:56 PM, Pete Wright  wrote:
>>
>> On 4/17/20 11:35 AM, Ryan Moeller wrote:
>>> OpenZFS brings many exciting features to FreeBSD, including:
>>>  * native encryption
>> Is there a good doc reference on available for using this?  I believe this
>> is zfs filesystem level encryption and not a replacement for our existing
>> full-disk-encryption scheme that currently works?
>
> I’m not aware of a good current doc for this. If anyone finds/writes
> something, please post it!
> There are some old resources you can find with a quick search that do a
> pretty good job of covering the basic ideas, but I think the exact syntax of
> commands may be slightly changed in the final implementation.
>
> The encryption is performed at a filesystem level (per-dataset).


You could find some initial doc and video about zfs encryption
on openzfs.org and youtube, and in some commit logs.
Therein was mentioned...

People are needed to volunteer to expand documentation on the
zfs crypto subject further in some document committed to openzfs
repo since users and orgs will want to know it when considering use.
Volunteers are also sought by openzfs to review the crypto itself.

Maybe there was already some central place made with further
current documentation about the zfs encryption topics since then?

https://www.youtube.com/watch?v=frnLiXclAMo openzfs encryption
https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view
https://www.youtube.com/watch?v=kFuo5bDj8C0 openzfs encryption cloud
https://drive.google.com/file/d/14uIZmJ48AfaQU4q69tED6MJf-RJhgwQR/view

It's dataset level, so GELI or GBDE etc are needed for full coverage,
perhaps even those two may not have yet received much or formal
review either, so there is always good volunteer opportunity there
to start with review of a potentially simpler cryptosystem like those.

zfs list, dataset snapshot names properties etc not covered.

zfs history not covered, many sensitive strings will end up in
there, including cutpaste password typos into commandline,
usernames, timestamps, etc...
and no tool exist to scrub overwrite history extents with random data,
and no option exists to turn keeping of
'user initiated events' or 'long format' off,
and ultimately no option exists to tell zpool create to
disable history keeping from the very start entirely.
So maybe users have to zero disks and pools along
with it just to scrub that.

zfs also exposes these variety of path and device names,
timestamps, etc in cleartext on disk structures in various
places, including configuration cachefile...
Some of those could could be NULLed or dummied
with new zpool create options for more security
restricted use cases.

There are other meta things and tools left exposed
such as potentially any plaintext meta in send/recv.

Another big metadata leak for environments and users that
ship, sell, embed, clone, distribute, fileshare, and backup,
their raw disks pools and usbs around to untrusted third parties...
is that zfs also puts hostnames and UUID type of unique
static meta and identifying things in cleartext on disk.
zfs thus needs options to allow users to set and use a NULL,
or generic dummy default, or random string, or chosen,
"hostname" for those from the very first zpool create command.

Most applications users use, including zfs, can today
consider ways in which metadata leaks could be removed
entirely, or at least optioned out for use under
high security restricted environments modes.
That could even involve considering trading off some
extra features not actually required for a basic mode
of functionality of the app.



(cc's for fyi inclusion about leaks, and as lists still haven't
been configured to support discreet bcc for that purpose,
which would also maintain nice headers for thread following.
Gmail breaks threads. zfsonlinux topicbox can't subscribe
without web. Drop non-relevant cc's from further replies.
Parent thread from freebsd current and stable lists was
Subject: OpenZFS port updated)

Re: OpenZFS imports, status update

[Warner Losh]

On Tue, Jun 8, 2021, 7:31 PM tech-lists  wrote:

> On Wed, Jun 09, 2021 at 03:14:01AM +0200, tue...@freebsd.org wrote:
> >Try: git remote prune freebsd
>
> unfortunately that didn't work either. In the end I rm -rf'd src and
> just cloned the repo again
>

You likely don't have freebsd as the remote origin. You need to replace it
with origin most likely.

Warner

> --
> J.
>

Re: OpenZFS imports, status update

[tech-lists]

On Wed, Jun 09, 2021 at 03:14:01AM +0200, tue...@freebsd.org wrote:

Try: git remote prune freebsd


unfortunately that didn't work either. In the end I rm -rf'd src and
just cloned the repo again
--
J.


signature.asc
Description: PGP signature

Re: OpenZFS imports, status update

[tech-lists]

On Tue, Jun 08, 2021 at 05:02:15PM -0600, Warner Losh wrote:

On Tue, Jun 8, 2021 at 4:58 PM Michael Tuexen <
michael.tue...@macmic.franken.de> wrote:


> On 9. Jun 2021, at 00:52, Warner Losh  wrote:
>
> Top posting...
>
> You can do a 'git remote purge freebsd' to reset the branch names, then
> you'll be able to do a normal pull.
Do you mean 'git remote prune freebsd'?



Yes.  Should have cut and paste instead of retyping it :(


root@desktop:/usr/src# git remote prune freebsd
fatal: 'freebsd' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

--
J.


signature.asc
Description: PGP signature

Re: OpenZFS imports, status update

[tuexen]

> On 9. Jun 2021, at 03:07, tech-lists  wrote:
> 
> Hi,
> 
> On Tue, Jun 08, 2021 at 04:52:43PM -0600, Warner Losh wrote:
>> Top posting...
>> 
>> You can do a 'git remote purge freebsd' to reset the branch names, then
>> you'll be able to do a normal pull.
> 
> This doesn't work for me. I use the anonssh method of updating the tree
> 
> % git remote -v
> originssh://anon...@git.freebsd.org/src.git (fetch)
> originssh://anon...@git.freebsd.org/src.git (push)
> 
> % gvs
> srctree last updated: Wed Jun 9 02:19:47 2021 +0200 version: 245848
> 
> root@desktop:/root# cd /usr/src
> 
> root@desktop:/usr/src# git version
> git version 2.31.1
> 
> root@desktop:/usr/src# git checkout stable/13
> Already on 'stable/13'
> Your branch is behind 'origin/stable/13' by 6922 commits, and can be 
> fast-forwarded.
>  (use "git pull" to update your local branch)
> 
> root@desktop:/usr/src# git remote purge freebsd
> error: Unknown subcommand: purge
Try: git remote prune freebsd

Best regards
Michael
> usage: git remote [-v | --verbose]
>   or: git remote add [-t ] [-m ] [-f] [--tags | --no-tags] 
> [--mirror=]  
>   or: git remote rename  
>   or: git remote remove 
>   or: git remote set-head  (-a | --auto | -d | --delete | )
>   or: git remote [-v | --verbose] show [-n] 
>   or: git remote prune [-n | --dry-run] 
>   or: git remote [-v | --verbose] update [-p | --prune] [( | 
> )...]
>   or: git remote set-branches [--add]  ...
>   or: git remote get-url [--push] [--all] 
>   or: git remote set-url [--push]   []
>   or: git remote set-url --add  
>   or: git remote set-url --delete  
> 
>-v, --verbose be verbose; must be placed before a subcommand
> 
> -- 
> J.

Re: OpenZFS imports, status update

[tech-lists]

Hi,

On Tue, Jun 08, 2021 at 04:52:43PM -0600, Warner Losh wrote:

Top posting...

You can do a 'git remote purge freebsd' to reset the branch names, then
you'll be able to do a normal pull.


This doesn't work for me. I use the anonssh method of updating the tree

% git remote -v
origin  ssh://anon...@git.freebsd.org/src.git (fetch)
origin  ssh://anon...@git.freebsd.org/src.git (push)

% gvs
srctree last updated: Wed Jun 9 02:19:47 2021 +0200 version: 245848

root@desktop:/root# cd /usr/src

root@desktop:/usr/src# git version
git version 2.31.1

root@desktop:/usr/src# git checkout stable/13
Already on 'stable/13'
Your branch is behind 'origin/stable/13' by 6922 commits, and can be 
fast-forwarded.
  (use "git pull" to update your local branch)

root@desktop:/usr/src# git remote purge freebsd
error: Unknown subcommand: purge
usage: git remote [-v | --verbose]
   or: git remote add [-t ] [-m ] [-f] [--tags | --no-tags] 
[--mirror=]  
   or: git remote rename  
   or: git remote remove 
   or: git remote set-head  (-a | --auto | -d | --delete | )
   or: git remote [-v | --verbose] show [-n] 
   or: git remote prune [-n | --dry-run] 
   or: git remote [-v | --verbose] update [-p | --prune] [( | 
)...]
   or: git remote set-branches [--add]  ...
   or: git remote get-url [--push] [--all] 
   or: git remote set-url [--push]   []
   or: git remote set-url --add  
   or: git remote set-url --delete  

-v, --verbose be verbose; must be placed before a subcommand

--
J.


signature.asc
Description: PGP signature

Re: Files in /etc containing empty VCSId header

[Ian Lepore]

On Tue, 2021-06-08 at 15:11 -0700, Rodney W. Grimes wrote:
> > On Tue, 8 Jun 2021 09:41:34 +
> > Mark Linimon  wrote:
> > 
> > > On Mon, Jun 07, 2021 at 01:58:01PM -0600, Ian Lepore wrote:
> > > > Sometimes it's a real interesting exercise to figure out where
> > > > a
> > > > file on your runtime system comes from in the source world.  
> 
> There is a command for that which does or use to do a pretty
> decent job of it called whereis(1).
> 

revolution > whereis ntp.conf
ntp.conf:
revolution > whereis netif
netif:
revolution > whereis services
services:

So how does that help me locate the origin of these files in the source
tree?

-- Ian

> > > 
> > > A tangential problem I trip over is "what is on this SD card?"
> > > 
> > > It generally takes me 5-10 minutes to remember:
> > > 
> > >   strings //boot/kernel/kernel | tail
> > > 
> > > AFIAK it's the only way to be sure; nothing in //* seems
> > > to have that data.
> > > 
> > > (Yes, in theory they all live in their own little box each of
> > > which is labeled but things happen ...)
> > > 
> > 
> > I use locate for this kind of search, e.g.
> > 
> > locate netoptions
> > /etc/rc.d/netoptions
> > /usr/src/libexec/rc/rc.d/netoptions
> > 
> > -- 
> > Gary Jennejohn
> > 
> > 
> 
> 

Re: OpenZFS imports, status update

[Warner Losh]

On Tue, Jun 8, 2021 at 4:58 PM Michael Tuexen <
michael.tue...@macmic.franken.de> wrote:

> > On 9. Jun 2021, at 00:52, Warner Losh  wrote:
> >
> > Top posting...
> >
> > You can do a 'git remote purge freebsd' to reset the branch names, then
> > you'll be able to do a normal pull.
> Do you mean 'git remote prune freebsd'?
>

Yes.  Should have cut and paste instead of retyping it :(

Warner


> Best regards
> Michael
> >
> > Thanks for the feedback
> >
> > Warner
> >
> > On Tue, Jun 8, 2021 at 4:10 PM Mark Millard via freebsd-git <
> > freebsd-...@freebsd.org> wrote:
> >
> >> Li-Wen Hsu  wrote (in part) on
> >> Date: Tue, 8 Jun 2021 06:29:51 +0800 :
> >>
> >>> The people have local branch tracking the original vendor/openzfs will
> >>> encounter issues like this whey doing `git pull`:
> >>>
> >>>error: cannot lock ref
> >>> 'refs/remotes/freebsd/vendor/openzfs/legacy':
> >>> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> >>> 'refs/remotes/freebsd/vendor/openzfs/legacy'
> >>>
> >>> The solution is update the upstream of the tracking branch:
> >>>
> >>>(change "freebsd" to "origin" if you use default remote name)
> >>>git update-ref -d refs/remotes/freebsd/vendor/openzfs
> >>>git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy
> >>
> >> I guess I misunderstood the meaning of the wording.
> >> I've never explicitly created a branch of or checked out
> >> anything from vendor/ or below. So I was not expecting
> >> anything. I do use main and stable/13 and releng/13.0
> >> explicitly in worktrees, no local commits involved for
> >> the context that this is from.
> >>
> >> Looks like some implicit initial configuration defaults
> >> mean "local branch tracking" of vendor/openzfs was
> >> implicitly/automatically in use in a way that fits
> >> the actual intent of the wording:
> >>
> >> # git fetch
> >> remote: Enumerating objects: 21551, done.
> >> remote: Counting objects: 100% (21551/21551), done.
> >> remote: Compressing objects: 100% (152/152), done.
> >> remote: Total 75711 (delta 21460), reused 21399 (delta 21399),
> pack-reused
> >> 54160
> >> Receiving objects: 100% (75711/75711), 46.02 MiB | 4.74 MiB/s, done.
> >> Resolving deltas: 100% (59104/59104), completed with 2449 local objects.
> >> From https://git.FreeBSD.org/src
> >>   e5f5b6a75c0a..f20893853e8e  main   ->
> >> freebsd/main
> >>   a4deddc0de32..53de482ad419  releng/11.4->
> >> freebsd/releng/11.4
> >>   f5ddeabc08e7..6e927d10c587  releng/12.2->
> >> freebsd/releng/12.2
> >>   8023e729a521..b74cdf1eceae  releng/13.0->
> >> freebsd/releng/13.0
> >>   dbb3df26a8a3..90e161ec6d11  stable/11  ->
> >> freebsd/stable/11
> >>   f146c0338c6c..0a41a2899eb2  stable/12  ->
> >> freebsd/stable/12
> >>   ddd0a8bffe35..04c4bd7f7b52  stable/13  ->
> >> freebsd/stable/13
> >>   1e02e5b0ba86..395770967c36  vendor/acpica  ->
> >> freebsd/vendor/acpica
> >> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy':
> >> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> >> 'refs/remotes/freebsd/vendor/openzfs/legacy'
> >> ! [new branch]vendor/openzfs/legacy  ->
> >> freebsd/vendor/openzfs/legacy  (unable to update local ref)
> >> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master':
> >> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> >> 'refs/remotes/freebsd/vendor/openzfs/master'
> >> ! [new branch]vendor/openzfs/master  ->
> >> freebsd/vendor/openzfs/master  (unable to update local ref)
> >> error: cannot lock ref
> >> 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release':
> >> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> >> 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
> >> ! [new branch]vendor/openzfs/zfs-2.1-release ->
> >> freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)
> >>   efec8223892b..40c7ff83e74e  vendor/wpa ->
> >> freebsd/vendor/wpa
> >> * [new tag]   vendor/acpica/20210604 ->
> >> vendor/acpica/20210604
> >>
> >> So, for now, I then did:
> >>
> >> # git update-ref -d refs/remotes/freebsd/vendor/openzfs
> >> # git fetch
> >> From https://git.FreeBSD.org/src
> >> * [new branch]vendor/openzfs/legacy  ->
> >> freebsd/vendor/openzfs/legacy
> >> * [new branch]vendor/openzfs/master  ->
> >> freebsd/vendor/openzfs/master
> >> * [new branch]vendor/openzfs/zfs-2.1-release ->
> >> freebsd/vendor/openzfs/zfs-2.1-release
> >>
> >> This order was apparently a mistake in some way because
> >> after it I then tried the other of the pair of commands
> >> and got:
> >>
> >> # git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy
> >> fatal: branch 'vendor/openzfs/legacy' does not exist
> 

Re: OpenZFS imports, status update

[Warner Losh]

Top posting...

You can do a 'git remote purge freebsd' to reset the branch names, then
you'll be able to do a normal pull.

Thanks for the feedback

Warner

On Tue, Jun 8, 2021 at 4:10 PM Mark Millard via freebsd-git <
freebsd-...@freebsd.org> wrote:

> Li-Wen Hsu  wrote (in part) on
> Date: Tue, 8 Jun 2021 06:29:51 +0800 :
>
> > The people have local branch tracking the original vendor/openzfs will
> > encounter issues like this whey doing `git pull`:
> >
> > error: cannot lock ref
> > 'refs/remotes/freebsd/vendor/openzfs/legacy':
> > 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> > 'refs/remotes/freebsd/vendor/openzfs/legacy'
> >
> > The solution is update the upstream of the tracking branch:
> >
> > (change "freebsd" to "origin" if you use default remote name)
> > git update-ref -d refs/remotes/freebsd/vendor/openzfs
> > git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy
>
> I guess I misunderstood the meaning of the wording.
> I've never explicitly created a branch of or checked out
> anything from vendor/ or below. So I was not expecting
> anything. I do use main and stable/13 and releng/13.0
> explicitly in worktrees, no local commits involved for
> the context that this is from.
>
> Looks like some implicit initial configuration defaults
> mean "local branch tracking" of vendor/openzfs was
> implicitly/automatically in use in a way that fits
> the actual intent of the wording:
>
> # git fetch
> remote: Enumerating objects: 21551, done.
> remote: Counting objects: 100% (21551/21551), done.
> remote: Compressing objects: 100% (152/152), done.
> remote: Total 75711 (delta 21460), reused 21399 (delta 21399), pack-reused
> 54160
> Receiving objects: 100% (75711/75711), 46.02 MiB | 4.74 MiB/s, done.
> Resolving deltas: 100% (59104/59104), completed with 2449 local objects.
> From https://git.FreeBSD.org/src
>e5f5b6a75c0a..f20893853e8e  main   ->
> freebsd/main
>a4deddc0de32..53de482ad419  releng/11.4->
> freebsd/releng/11.4
>f5ddeabc08e7..6e927d10c587  releng/12.2->
> freebsd/releng/12.2
>8023e729a521..b74cdf1eceae  releng/13.0->
> freebsd/releng/13.0
>dbb3df26a8a3..90e161ec6d11  stable/11  ->
> freebsd/stable/11
>f146c0338c6c..0a41a2899eb2  stable/12  ->
> freebsd/stable/12
>ddd0a8bffe35..04c4bd7f7b52  stable/13  ->
> freebsd/stable/13
>1e02e5b0ba86..395770967c36  vendor/acpica  ->
> freebsd/vendor/acpica
> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy':
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> 'refs/remotes/freebsd/vendor/openzfs/legacy'
>  ! [new branch]vendor/openzfs/legacy  ->
> freebsd/vendor/openzfs/legacy  (unable to update local ref)
> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master':
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> 'refs/remotes/freebsd/vendor/openzfs/master'
>  ! [new branch]vendor/openzfs/master  ->
> freebsd/vendor/openzfs/master  (unable to update local ref)
> error: cannot lock ref
> 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release':
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
>  ! [new branch]vendor/openzfs/zfs-2.1-release ->
> freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)
>efec8223892b..40c7ff83e74e  vendor/wpa ->
> freebsd/vendor/wpa
>  * [new tag]   vendor/acpica/20210604 ->
> vendor/acpica/20210604
>
> So, for now, I then did:
>
> # git update-ref -d refs/remotes/freebsd/vendor/openzfs
> # git fetch
> From https://git.FreeBSD.org/src
>  * [new branch]vendor/openzfs/legacy  ->
> freebsd/vendor/openzfs/legacy
>  * [new branch]vendor/openzfs/master  ->
> freebsd/vendor/openzfs/master
>  * [new branch]vendor/openzfs/zfs-2.1-release ->
> freebsd/vendor/openzfs/zfs-2.1-release
>
> This order was apparently a mistake in some way because
> after it I then tried the other of the pair of commands
> and got:
>
> # git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy
> fatal: branch 'vendor/openzfs/legacy' does not exist
>
> Yet what I see is:
>
> # git remote show freebsd
> * remote freebsd
>   Fetch URL: https://git.FreeBSD.org/src.git
>   Push  URL: https://git.FreeBSD.org/src.git
>   HEAD branch: main
>   Remote branches:
> main tracked
> refs/notes/commits   tracked
> . . .
> releng/13.0  tracked
> . . .
> stable/13tracked
> . . .
> vendor/openzfs/legacytracked
> vendor/openzfs/mastertracked
> vendor/openzfs/zfs-2.1-release   tracked
> . . .
>   

Re: ssh connections break with "Fssh_packet_write_wait" on 13 [SOLVED]

[tuexen]

> On 9. Jun 2021, at 00:20, Rodney W. Grimes  
> wrote:
> 
>> 
>> On Thu, 3 Jun 2021 15:09:06 +0200
>> Michael Gmelin  wrote:
>> 
>>> On Tue, 1 Jun 2021 13:47:47 +0200
>>> Michael Gmelin  wrote:
>>> 
 Hi,
 
 Since upgrading servers from 12.2 to 13.0, I get
 
 Fssh_packet_write_wait: Connection to 1.2.3.4 port 22: Broken pipe
 
 consistently, usually after about 11 idle minutes, that's with and
 without pf enabled. Client (11.4 in a VM) wasn't altered.
 
 Verbose logging (client and server side) doesn't show anything
 special when the connection breaks. In the past, QoS problems
 caused these disconnects, but I didn't see anything apparent
 changing between 12.2 and 13 in this respect.
 
 I did a test on a newly commissioned server to rule out other
 factors (so, same client connections, some routes, same
 everything). On 12.2 before the update: Connection stays open for
 hours. After the update (same server): connections breaks
 consistently after < 15 minutes (this is with unaltered
 configurations, no *AliveInterval configured on either side of the
 connection). 
>>> 
>>> I did a little bit more testing and realized that the problem goes
>>> away when I disable "Proportional Rate Reduction per RFC 6937" on the
>>> server side:
>>> 
>>> sysctl net.inet.tcp.do_prr=0
>>> 
>>> Keeping it on and enabling net.inet.tcp.do_prr_conservative doesn't
>>> fix the problem.
>>> 
>>> This seems to be specific to Parallels. After some more digging, I
>>> realized that Parallels Desktop's NAT daemon (prl_naptd) handles
>>> keep-alive between the VM and the external server on its own. There is
>>> no direct communication between the client and the server. This means:
>>> 
>>> - The NAT daemon starts sending keep-alive packages right away (not
>>> after the VM's net.inet.tcp.keepidle), every 75 seconds.
>>> - Keep-alive packages originating in the VM never reach the server.
>>> - Keep-alive originating on the server never reaches the VM.
>>> - Client and server basically do keep-alive with the nat daemon, not
>>> with each other.
>>> 
>>> It also seems like Parallels is filtering the tos field (so it's
>>> always 0x00), but that's unrelated.
>>> 
>>> I configured a bhyve VM running FreeBSD 11.4 on a separate laptop on
>>> the same network for comparison and is has no such issues.
>>> 
>>> Looking at TCP dump output on the server, this is what a keep-alive
>>> package sent by Parallels looks like:
>>> 
>>> 10:14:42.449681 IP (tos 0x0, ttl 64, id 15689, offset 0, flags
>>> [none], proto TCP (6), length 40)
>>>   192.168.1.1.58222 > 192.168.1.2.22: Flags [.], cksum x (correct),
>>>   seq 2534, ack 3851, win 4096, length 0
>>> 
>>> While those originating from the bhyve VM (after lowering
>>> net.inet.tcp.keepidle) look like this:
>>> 
>>> 12:18:43.105460 IP (tos 0x0, ttl 62, id 0, offset 0, flags [DF],
>>>   proto TCP (6), length 52)
>>>   192.168.1.3.57555 > 192.168.1.2.22: Flags [.], cksum x
>>>   (correct), seq 1780337696, ack 45831723, win 1026, options
>>>   [nop,nop,TS val 3003646737 ecr 3331923346], length 0
>>> 
>>> Like written above, once net.inet.tcp.do_prr is disabled, keepalive
>>> seems to be working just fine. Otherwise, Parallel's NAT daemon kills
>>> the connection, as its keep-alive requests are not answered (well,
>>> that's what I think is happening):
>>> 
>>> 10:19:43.614803 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF],
>>>   proto TCP (6), length 40)
>>>   192.168.1.1.58222 > 192.168.1.2.22: Flags [R.], cksum x (correct),
>>>   seq 2535, ack 3851, win 4096, length 0
>>> 
>>> The easiest way to work around the problem Client side is to configure
>>> ServerAliveInterval in ~/.ssh/config in the Client VM.
>>> 
>>> I'm curious though if this is basically a Parallels problem that has
>>> only been exposed by PRR being more correct (which is what I suspect),
>>> or if this is actually a FreeBSD problem.
>>> 
>> 
>> So, PRR probably was a red herring and the real reason that's happening
>> is that FreeBSD (since version 13[0]) by default discards packets
>> without timestamps for connections that formally had negotiated to have
>> them. This new behavior seems to be in line with RFC 7323, section
>> 3.2[1]:
>> 
>>   "Once TSopt has been successfully negotiated, that is both  and
>>contain TSopt, the TSopt MUST be sent in every non-
>>   segment for the duration of the connection, and SHOULD be sent in an
>>segment (see Section 5.2 for details)."
>> 
>> As it turns out, macOS does exactly this - send keep-alive packets
>> without a timestamp for connections that were negotiated to have them.
>> 
>> Under normal circumstances - ssh from macOS to a server running FreeBSD
>> 13 - this won't be noticed, since macOS uses the same default settings
>> as FreeBSD (2 hours idle time, 75 seconds intervals), so the server
>> side initiated keep-alive will save the connection before it has a
>> chance to break due to eight 

Re: ssh connections break with "Fssh_packet_write_wait" on 13 [SOLVED]

[Rodney W. Grimes]

> 
> On Thu, 3 Jun 2021 15:09:06 +0200
> Michael Gmelin  wrote:
> 
> > On Tue, 1 Jun 2021 13:47:47 +0200
> > Michael Gmelin  wrote:
> > 
> > > Hi,
> > > 
> > > Since upgrading servers from 12.2 to 13.0, I get
> > > 
> > >   Fssh_packet_write_wait: Connection to 1.2.3.4 port 22: Broken pipe
> > > 
> > > consistently, usually after about 11 idle minutes, that's with and
> > > without pf enabled. Client (11.4 in a VM) wasn't altered.
> > > 
> > > Verbose logging (client and server side) doesn't show anything
> > > special when the connection breaks. In the past, QoS problems
> > > caused these disconnects, but I didn't see anything apparent
> > > changing between 12.2 and 13 in this respect.
> > > 
> > > I did a test on a newly commissioned server to rule out other
> > > factors (so, same client connections, some routes, same
> > > everything). On 12.2 before the update: Connection stays open for
> > > hours. After the update (same server): connections breaks
> > > consistently after < 15 minutes (this is with unaltered
> > > configurations, no *AliveInterval configured on either side of the
> > > connection). 
> > 
> > I did a little bit more testing and realized that the problem goes
> > away when I disable "Proportional Rate Reduction per RFC 6937" on the
> > server side:
> > 
> >   sysctl net.inet.tcp.do_prr=0
> > 
> > Keeping it on and enabling net.inet.tcp.do_prr_conservative doesn't
> > fix the problem.
> > 
> > This seems to be specific to Parallels. After some more digging, I
> > realized that Parallels Desktop's NAT daemon (prl_naptd) handles
> > keep-alive between the VM and the external server on its own. There is
> > no direct communication between the client and the server. This means:
> > 
> > - The NAT daemon starts sending keep-alive packages right away (not
> >   after the VM's net.inet.tcp.keepidle), every 75 seconds.
> > - Keep-alive packages originating in the VM never reach the server.
> > - Keep-alive originating on the server never reaches the VM.
> > - Client and server basically do keep-alive with the nat daemon, not
> >   with each other.
> > 
> > It also seems like Parallels is filtering the tos field (so it's
> > always 0x00), but that's unrelated.
> > 
> > I configured a bhyve VM running FreeBSD 11.4 on a separate laptop on
> > the same network for comparison and is has no such issues.
> > 
> > Looking at TCP dump output on the server, this is what a keep-alive
> > package sent by Parallels looks like:
> > 
> >   10:14:42.449681 IP (tos 0x0, ttl 64, id 15689, offset 0, flags
> > [none], proto TCP (6), length 40)
> > 192.168.1.1.58222 > 192.168.1.2.22: Flags [.], cksum x (correct),
> > seq 2534, ack 3851, win 4096, length 0
> > 
> > While those originating from the bhyve VM (after lowering
> > net.inet.tcp.keepidle) look like this:
> > 
> >   12:18:43.105460 IP (tos 0x0, ttl 62, id 0, offset 0, flags [DF],
> > proto TCP (6), length 52)
> > 192.168.1.3.57555 > 192.168.1.2.22: Flags [.], cksum x
> > (correct), seq 1780337696, ack 45831723, win 1026, options
> > [nop,nop,TS val 3003646737 ecr 3331923346], length 0
> > 
> > Like written above, once net.inet.tcp.do_prr is disabled, keepalive
> > seems to be working just fine. Otherwise, Parallel's NAT daemon kills
> > the connection, as its keep-alive requests are not answered (well,
> > that's what I think is happening):
> > 
> >   10:19:43.614803 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF],
> > proto TCP (6), length 40)
> > 192.168.1.1.58222 > 192.168.1.2.22: Flags [R.], cksum x (correct),
> > seq 2535, ack 3851, win 4096, length 0
> > 
> > The easiest way to work around the problem Client side is to configure
> > ServerAliveInterval in ~/.ssh/config in the Client VM.
> > 
> > I'm curious though if this is basically a Parallels problem that has
> > only been exposed by PRR being more correct (which is what I suspect),
> > or if this is actually a FreeBSD problem.
> > 
> 
> So, PRR probably was a red herring and the real reason that's happening
> is that FreeBSD (since version 13[0]) by default discards packets
> without timestamps for connections that formally had negotiated to have
> them. This new behavior seems to be in line with RFC 7323, section
> 3.2[1]:
> 
> "Once TSopt has been successfully negotiated, that is both  and
>  contain TSopt, the TSopt MUST be sent in every non-
> segment for the duration of the connection, and SHOULD be sent in an
>  segment (see Section 5.2 for details)."
> 
> As it turns out, macOS does exactly this - send keep-alive packets
> without a timestamp for connections that were negotiated to have them.
> 
> Under normal circumstances - ssh from macOS to a server running FreeBSD
> 13 - this won't be noticed, since macOS uses the same default settings
> as FreeBSD (2 hours idle time, 75 seconds intervals), so the server
> side initiated keep-alive will save the connection before it has a
> chance to break due to eight consecutive unanswered 

Re: Files in /etc containing empty VCSId header

[Rodney W. Grimes]

> On Tue, 8 Jun 2021 09:41:34 +
> Mark Linimon  wrote:
> 
> > On Mon, Jun 07, 2021 at 01:58:01PM -0600, Ian Lepore wrote:
> > > Sometimes it's a real interesting exercise to figure out where a
> > > file on your runtime system comes from in the source world.  

There is a command for that which does or use to do a pretty
decent job of it called whereis(1).


> > 
> > A tangential problem I trip over is "what is on this SD card?"
> > 
> > It generally takes me 5-10 minutes to remember:
> > 
> >   strings //boot/kernel/kernel | tail
> > 
> > AFIAK it's the only way to be sure; nothing in //* seems
> > to have that data.
> > 
> > (Yes, in theory they all live in their own little box each of
> > which is labeled but things happen ...)
> > 
> 
> I use locate for this kind of search, e.g.
> 
> locate netoptions
> /etc/rc.d/netoptions
> /usr/src/libexec/rc/rc.d/netoptions
> 
> -- 
> Gary Jennejohn
> 
> 

-- 
Rod Grimes rgri...@freebsd.org

Re: OpenZFS imports, status update

[Mark Millard via freebsd-git]

Li-Wen Hsu  wrote (in part) on
Date: Tue, 8 Jun 2021 06:29:51 +0800 :

> The people have local branch tracking the original vendor/openzfs will
> encounter issues like this whey doing `git pull`:
> 
> error: cannot lock ref
> 'refs/remotes/freebsd/vendor/openzfs/legacy':
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
> 'refs/remotes/freebsd/vendor/openzfs/legacy'
> 
> The solution is update the upstream of the tracking branch:
> 
> (change "freebsd" to "origin" if you use default remote name)
> git update-ref -d refs/remotes/freebsd/vendor/openzfs
> git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy

I guess I misunderstood the meaning of the wording.
I've never explicitly created a branch of or checked out
anything from vendor/ or below. So I was not expecting
anything. I do use main and stable/13 and releng/13.0
explicitly in worktrees, no local commits involved for
the context that this is from.

Looks like some implicit initial configuration defaults
mean "local branch tracking" of vendor/openzfs was
implicitly/automatically in use in a way that fits
the actual intent of the wording:

# git fetch
remote: Enumerating objects: 21551, done.
remote: Counting objects: 100% (21551/21551), done.
remote: Compressing objects: 100% (152/152), done.
remote: Total 75711 (delta 21460), reused 21399 (delta 21399), pack-reused 54160
Receiving objects: 100% (75711/75711), 46.02 MiB | 4.74 MiB/s, done.
Resolving deltas: 100% (59104/59104), completed with 2449 local objects.
From https://git.FreeBSD.org/src
   e5f5b6a75c0a..f20893853e8e  main   -> freebsd/main
   a4deddc0de32..53de482ad419  releng/11.4-> 
freebsd/releng/11.4
   f5ddeabc08e7..6e927d10c587  releng/12.2-> 
freebsd/releng/12.2
   8023e729a521..b74cdf1eceae  releng/13.0-> 
freebsd/releng/13.0
   dbb3df26a8a3..90e161ec6d11  stable/11  -> 
freebsd/stable/11
   f146c0338c6c..0a41a2899eb2  stable/12  -> 
freebsd/stable/12
   ddd0a8bffe35..04c4bd7f7b52  stable/13  -> 
freebsd/stable/13
   1e02e5b0ba86..395770967c36  vendor/acpica  -> 
freebsd/vendor/acpica
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/legacy'
 ! [new branch]vendor/openzfs/legacy  -> 
freebsd/vendor/openzfs/legacy  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/master'
 ! [new branch]vendor/openzfs/master  -> 
freebsd/vendor/openzfs/master  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
 ! [new branch]vendor/openzfs/zfs-2.1-release -> 
freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)
   efec8223892b..40c7ff83e74e  vendor/wpa -> 
freebsd/vendor/wpa
 * [new tag]   vendor/acpica/20210604 -> 
vendor/acpica/20210604

So, for now, I then did:

# git update-ref -d refs/remotes/freebsd/vendor/openzfs
# git fetch
From https://git.FreeBSD.org/src
 * [new branch]vendor/openzfs/legacy  -> 
freebsd/vendor/openzfs/legacy
 * [new branch]vendor/openzfs/master  -> 
freebsd/vendor/openzfs/master
 * [new branch]vendor/openzfs/zfs-2.1-release -> 
freebsd/vendor/openzfs/zfs-2.1-release

This order was apparently a mistake in some way because
after it I then tried the other of the pair of commands
and got:

# git branch -u freebsd/vendor/openzfs/legacy vendor/openzfs/legacy
fatal: branch 'vendor/openzfs/legacy' does not exist

Yet what I see is:

# git remote show freebsd
* remote freebsd
  Fetch URL: https://git.FreeBSD.org/src.git
  Push  URL: https://git.FreeBSD.org/src.git
  HEAD branch: main
  Remote branches:
main tracked
refs/notes/commits   tracked
. . .
releng/13.0  tracked
. . .
stable/13tracked
. . .
vendor/openzfs/legacytracked
vendor/openzfs/mastertracked
vendor/openzfs/zfs-2.1-release   tracked
. . .
  Local branches configured for 'git pull':
mainmerges with remote main
releng/13.0 merges with remote releng/13.0
stable/13   merges with remote stable/13

So I'm not sure if I have anything that is messed up
or not. Nothing looks odd to me, other than the one
command's output.


Overall it seems that most people not doing something
special with an initial git configuration for FreeBSD
so have to deal with the issue at hand 

Re: ssh connections break with "Fssh_packet_write_wait" on 13 [SOLVED]

[Michael Gmelin]

On Thu, 3 Jun 2021 15:09:06 +0200
Michael Gmelin  wrote:

> On Tue, 1 Jun 2021 13:47:47 +0200
> Michael Gmelin  wrote:
> 
> > Hi,
> > 
> > Since upgrading servers from 12.2 to 13.0, I get
> > 
> >   Fssh_packet_write_wait: Connection to 1.2.3.4 port 22: Broken pipe
> > 
> > consistently, usually after about 11 idle minutes, that's with and
> > without pf enabled. Client (11.4 in a VM) wasn't altered.
> > 
> > Verbose logging (client and server side) doesn't show anything
> > special when the connection breaks. In the past, QoS problems
> > caused these disconnects, but I didn't see anything apparent
> > changing between 12.2 and 13 in this respect.
> > 
> > I did a test on a newly commissioned server to rule out other
> > factors (so, same client connections, some routes, same
> > everything). On 12.2 before the update: Connection stays open for
> > hours. After the update (same server): connections breaks
> > consistently after < 15 minutes (this is with unaltered
> > configurations, no *AliveInterval configured on either side of the
> > connection). 
> 
> I did a little bit more testing and realized that the problem goes
> away when I disable "Proportional Rate Reduction per RFC 6937" on the
> server side:
> 
>   sysctl net.inet.tcp.do_prr=0
> 
> Keeping it on and enabling net.inet.tcp.do_prr_conservative doesn't
> fix the problem.
> 
> This seems to be specific to Parallels. After some more digging, I
> realized that Parallels Desktop's NAT daemon (prl_naptd) handles
> keep-alive between the VM and the external server on its own. There is
> no direct communication between the client and the server. This means:
> 
> - The NAT daemon starts sending keep-alive packages right away (not
>   after the VM's net.inet.tcp.keepidle), every 75 seconds.
> - Keep-alive packages originating in the VM never reach the server.
> - Keep-alive originating on the server never reaches the VM.
> - Client and server basically do keep-alive with the nat daemon, not
>   with each other.
> 
> It also seems like Parallels is filtering the tos field (so it's
> always 0x00), but that's unrelated.
> 
> I configured a bhyve VM running FreeBSD 11.4 on a separate laptop on
> the same network for comparison and is has no such issues.
> 
> Looking at TCP dump output on the server, this is what a keep-alive
> package sent by Parallels looks like:
> 
>   10:14:42.449681 IP (tos 0x0, ttl 64, id 15689, offset 0, flags
> [none], proto TCP (6), length 40)
> 192.168.1.1.58222 > 192.168.1.2.22: Flags [.], cksum x (correct),
> seq 2534, ack 3851, win 4096, length 0
> 
> While those originating from the bhyve VM (after lowering
> net.inet.tcp.keepidle) look like this:
> 
>   12:18:43.105460 IP (tos 0x0, ttl 62, id 0, offset 0, flags [DF],
> proto TCP (6), length 52)
> 192.168.1.3.57555 > 192.168.1.2.22: Flags [.], cksum x
> (correct), seq 1780337696, ack 45831723, win 1026, options
> [nop,nop,TS val 3003646737 ecr 3331923346], length 0
> 
> Like written above, once net.inet.tcp.do_prr is disabled, keepalive
> seems to be working just fine. Otherwise, Parallel's NAT daemon kills
> the connection, as its keep-alive requests are not answered (well,
> that's what I think is happening):
> 
>   10:19:43.614803 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF],
> proto TCP (6), length 40)
> 192.168.1.1.58222 > 192.168.1.2.22: Flags [R.], cksum x (correct),
> seq 2535, ack 3851, win 4096, length 0
> 
> The easiest way to work around the problem Client side is to configure
> ServerAliveInterval in ~/.ssh/config in the Client VM.
> 
> I'm curious though if this is basically a Parallels problem that has
> only been exposed by PRR being more correct (which is what I suspect),
> or if this is actually a FreeBSD problem.
> 

So, PRR probably was a red herring and the real reason that's happening
is that FreeBSD (since version 13[0]) by default discards packets
without timestamps for connections that formally had negotiated to have
them. This new behavior seems to be in line with RFC 7323, section
3.2[1]:

"Once TSopt has been successfully negotiated, that is both  and
 contain TSopt, the TSopt MUST be sent in every non-
segment for the duration of the connection, and SHOULD be sent in an
 segment (see Section 5.2 for details)."

As it turns out, macOS does exactly this - send keep-alive packets
without a timestamp for connections that were negotiated to have them.

Under normal circumstances - ssh from macOS to a server running FreeBSD
13 - this won't be noticed, since macOS uses the same default settings
as FreeBSD (2 hours idle time, 75 seconds intervals), so the server
side initiated keep-alive will save the connection before it has a
chance to break due to eight consecutive unanswered keep-alives at the
client side.

This is different for ssh connections originating from a VM inside
Parallels, as connections created by prl_naptd will start sending tcp
keep-alives shortly after the connection becomes 

Re: 14-CURRENT/aarch64 build problem

[Michael Butler via freebsd-arm]

On 6/8/21 3:42 PM, Juraj Lutter wrote:




On 8 Jun 2021, at 21:38, bob prohaska  wrote:

FWIW, same problem seen here. In an added twist, git pull (hoping for
a fix) fails also:

root@www:/usr/src # git pull
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/legacy'
 From https://git.freebsd.org/src
! [new branch]  vendor/openzfs/legacy -> 
freebsd/vendor/openzfs/legacy  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/master'
! [new branch]  vendor/openzfs/master -> 
freebsd/vendor/openzfs/master  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
! [new branch]  vendor/openzfs/zfs-2.1-release -> 
freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)

Is this a problem at my end, or the server's end?


This already has been documented, as the old openzfs branch has been 
renamed/moved.

$ git remote prune freebsd
$ git pull

See: https://lists.freebsd.org/archives/freebsd-git/2021-June/15.html


Having pulled a completely new tree (rm -rf /usr/src/*; git clone), I 
get the following failure .. did I miss something?


===> cddl/lib/libicp_rescue (obj,all,install)
Building 
/usr/obj/usr/src/amd64.amd64/cddl/lib/libicp_rescue/libicp_rescue.so.3

building shared library libicp_rescue.so.3
Building /usr/obj/usr/src/amd64.amd64/cddl/lib/libicp_rescue/_libinstall
===> cddl/lib/libspl (obj,all,install)
make[4]: make[4]: don't know how to make atomic.S. Stop
`assert.c' is up to date.
`list.c' is up to date.
`mkdirp.c' is up to date.
`page.c' is up to date.
`timestamp.c' is up to date.
`zone.c' is up to date.
`include/sys/list.h' is up to date.
`include/sys/list_impl.h' is up to date.
`getexecname.c' is up to date.
`gethostid.c' is up to date.
`getmntany.c' is up to date.
`mnttab.c' is up to date.
`atomic.S' was not built (being made, type OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|DONE_ALLSRC|DONECYCLE)!
`afterdepend' was not built (deferred, type 
OP_DEPENDS|OP_NOTMAIN|OP_PHONY|OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|DONECYCLE)!
`afterdepend' has .ORDER dependency against .depend (deferred, type 
OP_DEPENDS|OP_NOPATH|OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|CYCLE|DONECYCLE)
`assert.o' was not built (unmade, type 
OP_DEPENDS|OP_NOPATH|OP_HAS_COMMANDS|OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|DONECYCLE)!
`list.o' was not built (unmade, type 
OP_DEPENDS|OP_NOPATH|OP_HAS_COMMANDS|OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|DONECYCLE)!

`
 [ .. etc. .. ]

imb

Re: 14-CURRENT/aarch64 build problem

[Juraj Lutter]

> On 8 Jun 2021, at 21:38, bob prohaska  wrote:
> 
> FWIW, same problem seen here. In an added twist, git pull (hoping for
> a fix) fails also:
> 
> root@www:/usr/src # git pull
> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy': 
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
> 'refs/remotes/freebsd/vendor/openzfs/legacy'
> From https://git.freebsd.org/src
> ! [new branch]  vendor/openzfs/legacy -> 
> freebsd/vendor/openzfs/legacy  (unable to update local ref)
> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master': 
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
> 'refs/remotes/freebsd/vendor/openzfs/master'
> ! [new branch]  vendor/openzfs/master -> 
> freebsd/vendor/openzfs/master  (unable to update local ref)
> error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release': 
> 'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
> 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
> ! [new branch]  vendor/openzfs/zfs-2.1-release -> 
> freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)
> 
> Is this a problem at my end, or the server's end?

This already has been documented, as the old openzfs branch has been 
renamed/moved.

$ git remote prune freebsd
$ git pull

See: https://lists.freebsd.org/archives/freebsd-git/2021-June/15.html

otis

—
Juraj Lutter
o...@freebsd.org

Re: 14-CURRENT/aarch64 build problem

[bob prohaska]

On Tue, Jun 08, 2021 at 09:15:37PM +0200, Juraj Lutter wrote:
> Hi,
> 
> I???m having problem to build recent 14-CURRENT/aarch64 as of 
> 6d2648bcaba9b14e2f5c76680f3e7608e1f125f4:
> 
> --- cddl/lib/libuutil__L ---
> make[4]: make[4]: don't know how to make uu_dprintf.c. Stop
> make[4]: make[4]: don't know how to make uu_open.c. Stop
> `uu_alloc.c' is up to date.
> `uu_avl.c' is up to date.
> `uu_dprintf.c' was not built (being made, type OP_DEPS_FOUND|OP_MARK, flags 
> REMAKE|DONE_WAIT|DONE_ALLSRC|DONECYCLE)!
> ???
> 
> The build is performed with pristine /usr/obj
> 

FWIW, same problem seen here. In an added twist, git pull (hoping for
a fix) fails also:

root@www:/usr/src # git pull
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/legacy': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/legacy'
>From https://git.freebsd.org/src
 ! [new branch]  vendor/openzfs/legacy -> 
freebsd/vendor/openzfs/legacy  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/master': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/master'
 ! [new branch]  vendor/openzfs/master -> 
freebsd/vendor/openzfs/master  (unable to update local ref)
error: cannot lock ref 'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release': 
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create 
'refs/remotes/freebsd/vendor/openzfs/zfs-2.1-release'
 ! [new branch]  vendor/openzfs/zfs-2.1-release -> 
freebsd/vendor/openzfs/zfs-2.1-release  (unable to update local ref)

Is this a problem at my end, or the server's end?

Thanks for reading,

bob prohaska
 

14-CURRENT/aarch64 build problem

[Juraj Lutter]

Hi,

I’m having problem to build recent 14-CURRENT/aarch64 as of 
6d2648bcaba9b14e2f5c76680f3e7608e1f125f4:

--- cddl/lib/libuutil__L ---
make[4]: make[4]: don't know how to make uu_dprintf.c. Stop
make[4]: make[4]: don't know how to make uu_open.c. Stop
`uu_alloc.c' is up to date.
`uu_avl.c' is up to date.
`uu_dprintf.c' was not built (being made, type OP_DEPS_FOUND|OP_MARK, flags 
REMAKE|DONE_WAIT|DONE_ALLSRC|DONECYCLE)!
…

The build is performed with pristine /usr/obj

Any hints, please?
Thanks
otis

—
Juraj Lutter
o...@freebsd.org

Re: Files in /etc containing empty VCSId header

[John Baldwin]

On 6/7/21 12:58 PM, Ian Lepore wrote:

On Mon, 2021-06-07 at 13:53 -0600, Warner Losh wrote:

On Mon, Jun 7, 2021 at 12:26 PM John Baldwin  wrote:


On 5/20/21 9:37 AM, Michael Gmelin wrote:

Hi,

After a binary update using freebsd-update, all files in /etc
contain
"empty" VCS Id headers, e.g.,

  $ head  /etc/nsswitch.conf
  #
  # nsswitch.conf(5) - name service switch configuration file
  # $FreeBSD$
  #
  group: compat
  group_compat: nis
  hosts: files dns
  netgroup: compat
  networks: files
  passwd: compat

After migrating to git, I would've expected those to contain
something
else or disappear completely. Is this expected and are there any
plans
to remove them completely?


I believe we might eventually remove them in the future, but doing
so
right now would introduce a lot of churn and the conversion to git
had enough other churn going on.



We'd planned on not removing things that might be merged to stable/12
since
those releases (12.3 only I think) will be built out of svn. We'll
likely
start to
remove things more widely as the stable/12 branch reaches EOL and
after.

Warner


It would be really nice if, instead of just deleting the $FreeBSD$
markers, they could be replaced with the path/filename of the file in
the source tree.  Sometimes it's a real interesting exercise to figure
out where a file on your runtime system comes from in the source world.
All the source tree layout changes that happened for packaged-base
makes it even more interesting.


My hope is that we un-break src/etc. :(  A few folks have looked at doing
that (notably Kyle).

--
John Baldwin

Re: kernel panic while copying files

[Gary Jennejohn]

On Tue, 8 Jun 2021 06:27:04 -0600
Warner Losh  wrote:

> On Tue, Jun 8, 2021 at 2:47 AM Gary Jennejohn  wrote:
> 
[snip old stuff]
> > Here the kgdb backtrace:
> >
> > Unread portion of the kernel message buffer:
> > panic: Duplicate free of 0xf800356b9000 from zone
> > 0xfe00dcbdd800(da_ccb) slab 0xf800356b9fd8(0)
> > cpuid = 8
> > time = 1623140519
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> > 0xfe00c5f398c0
> > vpanic() at vpanic+0x181/frame 0xfe00c5f39910
> > panic() at panic+0x43/frame 0xfe00c5f39970
> > uma_dbg_free() at uma_dbg_free+0x1e1/frame 0xfe00c5f399b0
> > uma_zfree_arg() at uma_zfree_arg+0x147/frame 0xfe00c5f39a00
> > camperiphdone() at camperiphdone+0x1b7/frame 0xfe00c5f39b20
> > xpt_done_process() at xpt_done_process+0x3dd/frame 0xfe00c5f39b60
> > xpt_done_td() at xpt_done_td+0xf5/frame 0xfe00c5f39bb0
> > fork_exit() at fork_exit+0x80/frame 0xfe00c5f39bf0
> > fork_trampoline() at fork_trampoline+0xe/frame 0xfe00c5f39bf0
> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
> > KDB: enter: panic
> >
> > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> > 55  __asm("movq %%gs:%P1,%0" : "=r" (td) : "n"
> > (offsetof(struct pcpu,
> > (kgdb) bt
> > #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> > #1  doadump (textdump=textdump@entry=0)
> > at /usr/src/sys/kern/kern_shutdown.c:399
> > #2  0x8040c39a in db_dump (dummy=,
> > dummy2=, dummy3=, dummy4=)
> > at /usr/src/sys/ddb/db_command.c:575
> > #3  0x8040c192 in db_command (last_cmdp=,
> > cmd_table=, dopager=dopager@entry=1)
> > at /usr/src/sys/ddb/db_command.c:482
> > #4  0x8040beed in db_command_loop ()
> > at /usr/src/sys/ddb/db_command.c:535
> > #5  0x8040f616 in db_trap (type=, code= > out>)  
> > at /usr/src/sys/ddb/db_main.c:270
> > #6  0x8066b1c4 in kdb_trap (type=type@entry=3, code=code@entry=0,
> > tf=, tf@entry=0xfe00c5f397f0)
> > at /usr/src/sys/kern/subr_kdb.c:727
> > #7  0x809a4e96 in trap (frame=0xfe00c5f397f0)
> > at /usr/src/sys/amd64/amd64/trap.c:604
> > #8  
> > #9  kdb_enter (why=0x80a61a23 "panic", msg=)
> > at /usr/src/sys/kern/subr_kdb.c:506
> > #10 0x806207a2 in vpanic (fmt=, ap=,
> > ap@entry=0xfe00c5f39950) at /usr/src/sys/kern/kern_shutdown.c:907
> > #11 0x80620533 in panic (
> > fmt=0x80d635c8  ".\024\244\200\377\377\377\377")
> > at /usr/src/sys/kern/kern_shutdown.c:843
> > #12 0x808e12b1 in uma_dbg_free (zone=0xfe00dcbdd800,
> > slab=0xf800356b9fd8, item=0xf800356b9000)
> > at /usr/src/sys/vm/uma_core.c:5664
> > #13 0x808d9de7 in item_dtor (zone=0xfe00dcbdd800,
> > item=0xf800356b9000, size=544, udata=0x0, skip=SKIP_NONE)
> > at /usr/src/sys/vm/uma_core.c:3418
> > #14 uma_zfree_arg (zone=0xfe00dcbdd800, item=0xf800356b9000,
> > udata=udata@entry=0x0) at /usr/src/sys/vm/uma_core.c:4374
> > #15 0x802da503 in uma_zfree (zone=0x80d635c8 ,
> > item=0x200) at /usr/src/sys/vm/uma.h:404
> >  
> 
> OK. This is a bad stack trace. camperiphdone doesn't call uma_zfree()...
> It does call
> xpt_free_ccb, though, and that's likely what's going wrong. And that
> matches the line
> numbers. Most likely this is llvm's tail call optimizations...  Can you
> compile the kernel
> either -O0 or with -fno-optimize-sibling-calls? That will give a better
> call stack.
> 
> However, it's likely the new UMA stuff trasz committed (or it's providing
> better
> diagnostics than the old malloc based code which seems more likely) that
> can be
> disabled by the tunable kern.cam.da.enable_uma_ccbs=0.
> 
> The lines in question:
> saved_ccb = (union ccb *)done_ccb->ccb_h.saved_ccb_ptr;
> bcopy(saved_ccb, done_ccb, sizeof(*done_ccb));
> xpt_free_ccb(saved_ccb);
> 
> So we overwrite the done_ccb with the saved_ccb's contents and then free
> the saved ccb.
> That's likely OKish, though.
> 
> We copy entire CCBs around in this code a lot, and I've not traced through
> it. But we're
> sending a scsi start unit in response to some error that is being reported
> via cam_periph_error()
> 
> #16 0x802d9117 in camperiphdone (periph=0xf800061e2c00,
> > done_ccb=0xf800355d6cc0) at /usr/src/sys/cam/cam_periph.c:1427
> > #17 0x802dfebd in xpt_done_process (ccb_h=0xf800355d6cc0)
> > at /usr/src/sys/cam/cam_xpt.c:5491
> > #18 0x802e1ec5 in xpt_done_td (
> > arg=arg@entry=0x80d33d80 )
> > at /usr/src/sys/cam/cam_xpt.c:5546
> > #19 0x805dad80 in fork_exit (callout=0x802e1dd0
> > ,
> > arg=0x80d33d80 , frame=0xfe00c5f39c00)
> > at /usr/src/sys/kern/kern_fork.c:1083
> > #20 
> >
> > Apparently caused by recent changes to CAM.
> >
> > Let me know if you want more information.
> >  
> 
> what's 

Re: kernel panic while copying files

[Warner Losh]

On Tue, Jun 8, 2021 at 8:42 AM Gary Jennejohn  wrote:

> On Tue, 8 Jun 2021 06:48:19 -0600
> Warner Losh  wrote:
>
> > Sorry to reply to myself... had a thought as my brain rested while making
> > tea...
> >
> > I think we may need to consider reverting (or at least not yet enabling)
> > the uma stuff.
> >
>
> I tested and enabled the UMA CCB stuff immediately after trasz had
> committed it.  I was able to copy files panic-free over USB until
> recently AFAICR.
>
> I also have had the kern.cam.ada.enable_uma_ccbs=1 set since
> then and have never seen a problem there.  Only with USB.
>

Yes. This specific bug only affects SCSI. And it only affects it when
there's an error that requires a restart. I've not yet had the time to do
an audit for where else the copying is done...


> I'll try booting a new kernel with the uma_ccb sysctl's set to 0
> and see what happens.
>
> BTW I now have a kernel compiled with -O0 ready to test.
>

Great!

Warner

Re: Files in /etc containing empty VCSId header

[Ian Lepore]

On Tue, 2021-06-08 at 13:47 +0200, Gary Jennejohn wrote:
> On Tue, 8 Jun 2021 09:41:34 +
> Mark Linimon  wrote:
> 
> > On Mon, Jun 07, 2021 at 01:58:01PM -0600, Ian Lepore wrote:
> > > Sometimes it's a real interesting exercise to figure out where a
> > > file on your runtime system comes from in the source world.  
> > 
> > A tangential problem I trip over is "what is on this SD card?"
> > 
> > It generally takes me 5-10 minutes to remember:
> > 
> >   strings //boot/kernel/kernel | tail
> > 
> > AFIAK it's the only way to be sure; nothing in //* seems
> > to have that data.
> > 
> > (Yes, in theory they all live in their own little box each of
> > which is labeled but things happen ...)
> > 
> 
> I use locate for this kind of search, e.g.
> 
> locate netoptions
> /etc/rc.d/netoptions
> /usr/src/libexec/rc/rc.d/netoptions
> 

Not a useful technique on my machine:

revolution > locate netoptions | wc -l
 192

I have dozens of build chroots for various products for $work.

-- Ian

Re: kernel panic while copying files

[Gary Jennejohn]

On Tue, 8 Jun 2021 06:48:19 -0600
Warner Losh  wrote:

> Sorry to reply to myself... had a thought as my brain rested while making
> tea...
> 
> I think we may need to consider reverting (or at least not yet enabling)
> the uma stuff.
> 

I tested and enabled the UMA CCB stuff immediately after trasz had
committed it.  I was able to copy files panic-free over USB until
recently AFAICR.

I also have had the kern.cam.ada.enable_uma_ccbs=1 set since
then and have never seen a problem there.  Only with USB.

I'll try booting a new kernel with the uma_ccb sysctl's set to 0
and see what happens.

BTW I now have a kernel compiled with -O0 ready to test.

[snip lots of extraneous stuff]

-- 
Gary Jennejohn

Re: [HEADS UP] Rename of the vendor/openzfs branch

[Li-Wen Hsu]

On Tue, Jun 8, 2021 at 7:45 PM Li-Wen Hsu  wrote:
>
> Hello,
>
> As mentioned in the "OpenZFS imports, status update":
>
> https://lists.freebsd.org/archives/freebsd-git/2021-June/13.html
>
> We're going to rename the current openzfs vendor branch,
> vendor/openzfs, to vendor/openzfs/legacy

The renaming has been done:

https://cgit.freebsd.org/src/log/?h=vendor/openzfs/legacy

If you have a local branch tracking vendor/openzfs, please refer to
the previous mail for migration.

https://lists.freebsd.org/archives/freebsd-git/2021-June/15.html

Best,
Li-Wen

Re: kernel panic while copying files

[Warner Losh]

Sorry to reply to myself... had a thought as my brain rested while making
tea...

I think we may need to consider reverting (or at least not yet enabling)
the uma stuff.

On Tue, Jun 8, 2021 at 6:27 AM Warner Losh  wrote:

>
>
> On Tue, Jun 8, 2021 at 2:47 AM Gary Jennejohn 
> wrote:
>
>> On Mon, 7 Jun 2021 16:54:11 -0400
>> Mark Johnston  wrote:
>>
>> > On Mon, Jun 07, 2021 at 11:01:09AM +0200, Gary Jennejohn wrote:
>> > > I've seen this panic three times in the last two days:
>> > >
>> > > [first panic]
>> > > Unread portion of the kernel message buffer:
>> > >
>> > >
>> > > Fatal trap 12: page fault while in kernel mode
>> > > cpuid = 3; apic id = 03
>> > > fault virtual address   = 0x801118000
>> > > fault code  = supervisor write data, page not present
>> > > instruction pointer = 0x20:0x808d2212
>> > > stack pointer   = 0x28:0xfe00dbc8c760
>> > > frame pointer   = 0x28:0xfe00dbc8c7a0
>> > > code segment= base 0x0, limit 0xf, type 0x1b
>> > > = DPL 0, pres 1, long 1, def32 0, gran 1
>> > > processor eflags= interrupt enabled, resume, IOPL = 0
>> > > current process = 28 (dom0)
>> > > trap number = 12
>> > > panic: page fault
>> > > cpuid = 3
>> > > time = 1622963058
>> > > KDB: stack backtrace:
>> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>> 0xfe00dbc8c410
>> > > vpanic() at vpanic+0x181/frame 0xfe00dbc8c460
>> > > panic() at panic+0x43/frame 0xfe00dbc8c4c0
>> > > trap_fatal() at trap_fatal+0x387/frame 0xfe00dbc8c520
>> > > trap_pfault() at trap_pfault+0x4f/frame 0xfe00dbc8c580
>> > > trap() at trap+0x253/frame 0xfe00dbc8c690
>> > > calltrap() at calltrap+0x8/frame 0xfe00dbc8c690
>> > > --- trap 0xc, rip = 0x808d2212, rsp = 0xfe00dbc8c760, rbp
>> = 0xfe00dbc8c7a0 ---
>> > > zone_release() at zone_release+0x1f2/frame 0xfe00dbc8c7a0
>> > > bucket_drain() at bucket_drain+0xda/frame 0xfe00dbc8c7d0
>> > > bucket_cache_reclaim_domain() at
>> bucket_cache_reclaim_domain+0x30a/frame 0xfe00dbc8c830
>> > > zone_reclaim() at zone_reclaim+0x162/frame 0xfe00dbc8c880
>> > > uma_reclaim_domain() at uma_reclaim_domain+0xa2/frame
>> 0xfe00dbc8c8b0
>> > > vm_pageout_worker() at vm_pageout_worker+0x41e/frame
>> 0xfe00dbc8cb70
>> > > vm_pageout() at vm_pageout+0x21e/frame 0xfe00dbc8cbb0
>> > > fork_exit() at fork_exit+0x7e/frame 0xfe00dbc8cbf0
>> > > fork_trampoline() at fork_trampoline+0xe/frame 0xfe00dbc8cbf0
>> > > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
>> > > KDB: enter: panic
>> > >
>> > > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
>> > > 55   __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
>> pcpu,
>> > >pc_curthread)));
>> > >
>> > > One difference was that in the second and third panics the fault
>> virtual
>> > > address was 0x0.  But the backtrace was the same.
>> > >
>> > > Relevant info from the info.x files:
>> > > Architecture: amd64
>> > > Architecture Version: 2
>> > > Version String: FreeBSD 14.0-CURRENT #33 main-n247184-1970d693039:
>> Sat Jun
>> > > 5 09:58:55 CEST 2021
>> > >
>> > > CPU: AMD Ryzen 5 1600 Six-Core Processor (3194.09-MHz
>> K8-class CPU)
>> > >   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1
>> Stepping=1
>> > >   AMD Features=0x2e500800
>> > >   AMD
>> Features2=0x35c233ff
>> > >   AMD Extended Feature Extensions ID
>> EBX=0x1007
>> > >
>> > > I have 16GiB of memory in the box.
>> > >
>> > > The panic occurred while copying files from an internal SATA SSD to a
>> > > SATA 8TB disk in an external USB3 docking station.  The panic seems to
>> > > occur quite quickly, after only a few files have been copied.
>> > >
>> > > swap is on a different internal disk.
>> > >
>> > > I can poke around in the crash dumps with kgdb if anyone wants more
>> > > information.
>> >
>> > Are you running with invariants configured in the kernel?  If not,
>> > please try to reproduce this in a kernel with
>> >
>> > options INVARIANT_SUPPORT
>> > options INVARIANTS
>> >
>> > configured.
>> >
>> > A stack trace with line numbers would also be helpful.
>>
>> Thanks for the hint.  After enabling INVARIANTS the kernel panics as
>> soon I turn on the external USB3 disk.  No user disk access required.
>>
>> Version String: FreeBSD 14.0-CURRENT #34 main-n247239-f570a6723e1: Tue Jun
>> 8 09:34:32 CEST 2021
>>
>> Here the kgdb backtrace:
>>
>> Unread portion of the kernel message buffer:
>> panic: Duplicate free of 0xf800356b9000 from zone
>> 0xfe00dcbdd800(da_ccb) slab 0xf800356b9fd8(0)
>> cpuid = 8
>> time = 1623140519
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>> 0xfe00c5f398c0
>> vpanic() at vpanic+0x181/frame 0xfe00c5f39910
>> panic() at panic+0x43/frame 0xfe00c5f39970
>> uma_dbg_free() at uma_dbg_free+0x1e1/frame 0xfe00c5f399b0
>> uma_zfree_arg() at 

Re: kernel panic while copying files

[Warner Losh]

On Tue, Jun 8, 2021 at 2:47 AM Gary Jennejohn  wrote:

> On Mon, 7 Jun 2021 16:54:11 -0400
> Mark Johnston  wrote:
>
> > On Mon, Jun 07, 2021 at 11:01:09AM +0200, Gary Jennejohn wrote:
> > > I've seen this panic three times in the last two days:
> > >
> > > [first panic]
> > > Unread portion of the kernel message buffer:
> > >
> > >
> > > Fatal trap 12: page fault while in kernel mode
> > > cpuid = 3; apic id = 03
> > > fault virtual address   = 0x801118000
> > > fault code  = supervisor write data, page not present
> > > instruction pointer = 0x20:0x808d2212
> > > stack pointer   = 0x28:0xfe00dbc8c760
> > > frame pointer   = 0x28:0xfe00dbc8c7a0
> > > code segment= base 0x0, limit 0xf, type 0x1b
> > > = DPL 0, pres 1, long 1, def32 0, gran 1
> > > processor eflags= interrupt enabled, resume, IOPL = 0
> > > current process = 28 (dom0)
> > > trap number = 12
> > > panic: page fault
> > > cpuid = 3
> > > time = 1622963058
> > > KDB: stack backtrace:
> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfe00dbc8c410
> > > vpanic() at vpanic+0x181/frame 0xfe00dbc8c460
> > > panic() at panic+0x43/frame 0xfe00dbc8c4c0
> > > trap_fatal() at trap_fatal+0x387/frame 0xfe00dbc8c520
> > > trap_pfault() at trap_pfault+0x4f/frame 0xfe00dbc8c580
> > > trap() at trap+0x253/frame 0xfe00dbc8c690
> > > calltrap() at calltrap+0x8/frame 0xfe00dbc8c690
> > > --- trap 0xc, rip = 0x808d2212, rsp = 0xfe00dbc8c760, rbp
> = 0xfe00dbc8c7a0 ---
> > > zone_release() at zone_release+0x1f2/frame 0xfe00dbc8c7a0
> > > bucket_drain() at bucket_drain+0xda/frame 0xfe00dbc8c7d0
> > > bucket_cache_reclaim_domain() at
> bucket_cache_reclaim_domain+0x30a/frame 0xfe00dbc8c830
> > > zone_reclaim() at zone_reclaim+0x162/frame 0xfe00dbc8c880
> > > uma_reclaim_domain() at uma_reclaim_domain+0xa2/frame
> 0xfe00dbc8c8b0
> > > vm_pageout_worker() at vm_pageout_worker+0x41e/frame 0xfe00dbc8cb70
> > > vm_pageout() at vm_pageout+0x21e/frame 0xfe00dbc8cbb0
> > > fork_exit() at fork_exit+0x7e/frame 0xfe00dbc8cbf0
> > > fork_trampoline() at fork_trampoline+0xe/frame 0xfe00dbc8cbf0
> > > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
> > > KDB: enter: panic
> > >
> > > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> > > 55   __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
> pcpu,
> > >pc_curthread)));
> > >
> > > One difference was that in the second and third panics the fault
> virtual
> > > address was 0x0.  But the backtrace was the same.
> > >
> > > Relevant info from the info.x files:
> > > Architecture: amd64
> > > Architecture Version: 2
> > > Version String: FreeBSD 14.0-CURRENT #33 main-n247184-1970d693039: Sat
> Jun
> > > 5 09:58:55 CEST 2021
> > >
> > > CPU: AMD Ryzen 5 1600 Six-Core Processor (3194.09-MHz
> K8-class CPU)
> > >   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1
> Stepping=1
> > >   AMD Features=0x2e500800
> > >   AMD
> Features2=0x35c233ff
> > >   AMD Extended Feature Extensions ID
> EBX=0x1007
> > >
> > > I have 16GiB of memory in the box.
> > >
> > > The panic occurred while copying files from an internal SATA SSD to a
> > > SATA 8TB disk in an external USB3 docking station.  The panic seems to
> > > occur quite quickly, after only a few files have been copied.
> > >
> > > swap is on a different internal disk.
> > >
> > > I can poke around in the crash dumps with kgdb if anyone wants more
> > > information.
> >
> > Are you running with invariants configured in the kernel?  If not,
> > please try to reproduce this in a kernel with
> >
> > options INVARIANT_SUPPORT
> > options INVARIANTS
> >
> > configured.
> >
> > A stack trace with line numbers would also be helpful.
>
> Thanks for the hint.  After enabling INVARIANTS the kernel panics as
> soon I turn on the external USB3 disk.  No user disk access required.
>
> Version String: FreeBSD 14.0-CURRENT #34 main-n247239-f570a6723e1: Tue Jun
> 8 09:34:32 CEST 2021
>
> Here the kgdb backtrace:
>
> Unread portion of the kernel message buffer:
> panic: Duplicate free of 0xf800356b9000 from zone
> 0xfe00dcbdd800(da_ccb) slab 0xf800356b9fd8(0)
> cpuid = 8
> time = 1623140519
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfe00c5f398c0
> vpanic() at vpanic+0x181/frame 0xfe00c5f39910
> panic() at panic+0x43/frame 0xfe00c5f39970
> uma_dbg_free() at uma_dbg_free+0x1e1/frame 0xfe00c5f399b0
> uma_zfree_arg() at uma_zfree_arg+0x147/frame 0xfe00c5f39a00
> camperiphdone() at camperiphdone+0x1b7/frame 0xfe00c5f39b20
> xpt_done_process() at xpt_done_process+0x3dd/frame 0xfe00c5f39b60
> xpt_done_td() at xpt_done_td+0xf5/frame 0xfe00c5f39bb0
> fork_exit() at fork_exit+0x80/frame 0xfe00c5f39bf0
> fork_trampoline() at fork_trampoline+0xe/frame 

Re: Files in /etc containing empty VCSId header

[Gary Jennejohn]

On Tue, 8 Jun 2021 09:41:34 +
Mark Linimon  wrote:

> On Mon, Jun 07, 2021 at 01:58:01PM -0600, Ian Lepore wrote:
> > Sometimes it's a real interesting exercise to figure out where a
> > file on your runtime system comes from in the source world.  
> 
> A tangential problem I trip over is "what is on this SD card?"
> 
> It generally takes me 5-10 minutes to remember:
> 
>   strings //boot/kernel/kernel | tail
> 
> AFIAK it's the only way to be sure; nothing in //* seems
> to have that data.
> 
> (Yes, in theory they all live in their own little box each of
> which is labeled but things happen ...)
> 

I use locate for this kind of search, e.g.

locate netoptions
/etc/rc.d/netoptions
/usr/src/libexec/rc/rc.d/netoptions

-- 
Gary Jennejohn

[HEADS UP] Rename of the vendor/openzfs branch

[Li-Wen Hsu]

Hello,

As mentioned in the "OpenZFS imports, status update":

https://lists.freebsd.org/archives/freebsd-git/2021-June/13.html

We're going to rename the current openzfs vendor branch,
vendor/openzfs, to vendor/openzfs/legacy
and import directly the master and zfs-2.1-release branches from the
upstream OpenZFS to vendor/openzfs/* for merging to our main and
stable/13 branches. The details can be found in the mail above.

The people have local branch tracking the original vendor/openzfs may
encounter error message like this whey doing `git pull`:

error: cannot lock ref
'refs/remotes/freebsd/vendor/openzfs/legacy':
'refs/remotes/freebsd/vendor/openzfs' exists; cannot create
'refs/remotes/freebsd/vendor/openzfs/legacy'

The solution is update the upstream of the tracking branch:

(change "freebsd" to "origin" if you use the default remote name)
git remote prune freebsd
(on your tracking branch)
git branch -u freebsd/vendor/openzfs/legacy

If you don't want the first command prune all the branches not
existing in the remote repository, use this:

git update-ref -d refs/remotes/freebsd/vendor/openzfs
(also, change "freebsd" to "origin" if you use the default remote name)

After that you can do `git pull` or `git fetch` as usual.

If you have any questions, please post on -git@ list.

Best,
Li-Wen

Re: kernel panic while copying files

[Gary Jennejohn]

On Tue, 8 Jun 2021 11:04:33 +0200
Mateusz Guzik  wrote:

> Given how easy it is to reproduce perhaps you can spend a little bit
> of time narrowing it down to a specific commit. You can do it with
> git-bisect.
> 

Ok, I'll give it a try.

-- 
Gary Jennejohn

Re: kernel panic while copying files

[Hans Petter Selasky]

On 6/8/21 1:34 PM, Gary Jennejohn wrote:

Fields in the ccb like periph_name, unit_number and dev_name are filled
with zeroes.


Smells like a double free, like the panic message indicates, but would 
be nice to know exactly which driver is doing this, if it is "ATA" or 
"UMASS", so to speak.


Maybe you need to do a quick bisect, like suggested.

--HPS

Re: kernel panic while copying files

[Gary Jennejohn]

On Tue, 8 Jun 2021 11:20:37 +0200
Hans Petter Selasky  wrote:

> On 6/8/21 11:04 AM, Mateusz Guzik wrote:
> > Apparently caused by recent changes to CAM.
> > 
> > Let me know if you want more information.  
> 
> Maybe you can print the *ccb being freed and figure out which device
> it belongs to.
> 

I'm now running a kernel without INVARIANTS, so I can check:

Jun  8 13:23:52 ernst kernel: ugen2.4:  at usbus2
Jun  8 13:23:52 ernst kernel: umass0 on uhub5
Jun  8 13:23:52 ernst kernel: umass0:  on usbus2
Jun  8 13:23:52 ernst kernel: umass0:  SCSI over Bulk-Only; quirks = 0xc101
Jun  8 13:23:52 ernst kernel: umass0:6:0: Attached to scbus6
Jun  8 13:24:37 ernst kernel: da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
Jun  8 13:24:37 ernst kernel: da0:  Fixed Direct Access 
SPC-4 SCSI device
Jun  8 13:24:37 ernst kernel: da0: Serial Number 0001
Jun  8 13:24:37 ernst kernel: da0: 400.000MB/s transfers
Jun  8 13:24:37 ernst kernel: da0: 7630885MB (15628053168 512 byte sectors)
Jun  8 13:24:37 ernst kernel: da0: quirks=0x2

The only USB device which I turned on.

Fields in the ccb like periph_name, unit_number and dev_name are filled
with zeroes.

The ccb is enormous and really hard to parse.

-- 
Gary Jennejohn

Re: What happen to mailing list archives?

[Baptiste Daroussin]

8 juin 2021 13:15:50 Michael Gmelin :

>
>
> On Mon, 7 Jun 2021 22:35:20 +0200
> Baptiste Daroussin  wrote:
>
>> On Mon, Jun 07, 2021 at 12:30:46AM -0700, Mark Millard wrote:
>>> On 2021-Jun-6, at 13:25, Mark Millard  wrote:
>>>  
 Baptiste Daroussin  wrote on
 Date: Sun, 6 Jun 2021 10:53:49 +0200 :
  
> What has happended:
> plan A: we migrated everything off mailman/pipermail seamlessly
> with redirection and so on. We patched the new archiver to
> produce the same file names has pipermail
>
> Plan A worked fine up to a limit, there was plenty of hand
> edition in the past, we we decided to move to plan B which is
> what is happening now.
>
> Plan B: We keep a frozen version of the archives up to the
> migration date under the pipermail directory and have the new
> archives created in the archives directory.
>
> All the pipermail archives have been restored as they were. The
> new archives receives in their index a new link to point people
> to the pipermails archive if looking for older archives.
>
> this has been done a couple of hours ago (before Steve emails)
> during a window, of ~ 10 hours, the mailing lists which slow
> traffic aka the one which didn't received any email since the
> migration ended up with an empty "archives" directory (aka a
> 404), a file with explanation and redirection to pipermail has
> been installed there.
>
> Some work is still needed for the mailing lists which has been
> transformed as readonly, this will be done in the next couple of
> days 

 It is too bad that a reference to a "no examples yet"
 month, such as, (at the time I write this):

 https://lists.freebsd.org/archives/freebsd-toolchain/2021-June/date.html

 does not show at least (Date view specific example):

   • Other periods:[ Previous, Date view ] [ List of Folders
 ]
   • Other: [ Actives mailing lists ] [ Archives from
 mailman's time ]

 when there are prior months available in
 https://lists.freebsd.org/archives/ or show at least just:

   • Other: [ Actives mailing lists ] [ Archives from
 mailman's time ]

 when no prior months are available there.
  
>>>
>>> Looks like there are missing months.
>>>
>>> Using freebsd-hackers as an example:
>>>
>>> https://lists.freebsd.org/archives/freebsd-hackers/index.html
>>> shows the oldest month being "May 2021".
>>>
>>> But . . .
>>>
>>> https://lists.freebsd.org/pipermail/freebsd-hackers/
>>> shows the most recent month being "September 2020".
>>>
>>> So: 2020-Oct through 2021-Apr are completely missing.
>>>
>>> Then, going for more detail for 2020-Sep and 2021-May . . .
>>>
>>> https://lists.freebsd.org/archives/freebsd-hackers/2021-May/date.html
>>> shows "Starting Tue May 18 2021 - 21:07:44 UTC".
>>>
>>> https://lists.freebsd.org/pipermail/freebsd-hackers/2020-September/date.html
>>> shows "Ending: Tue Sep 15 14:12:20 UTC 2020".
>>>
>>> So there are about 2 more half-months missing.
>>>
>>> Some other lists have other date ranges, some similar,
>>> some not. 
>>
>> This missing month are being populated right now
>
> Will historic links still work though (so that old references work and
> current google results are ok)?
>
> Just stumbled over this one:
> https://lists.freebsd.org/archives/freebsd-jail/2017-March/003360.html
>
Old reference yes, not the one that lived a couple of weeks like this one

Bapt

Re: What happen to mailing list archives?

[Michael Gmelin]

On Mon, 7 Jun 2021 22:35:20 +0200
Baptiste Daroussin  wrote:

> On Mon, Jun 07, 2021 at 12:30:46AM -0700, Mark Millard wrote:
> > On 2021-Jun-6, at 13:25, Mark Millard  wrote:
> >   
> > > Baptiste Daroussin  wrote on
> > > Date: Sun, 6 Jun 2021 10:53:49 +0200 :
> > >   
> > >> What has happended:
> > >> plan A: we migrated everything off mailman/pipermail seamlessly
> > >> with redirection and so on. We patched the new archiver to
> > >> produce the same file names has pipermail
> > >> 
> > >> Plan A worked fine up to a limit, there was plenty of hand
> > >> edition in the past, we we decided to move to plan B which is
> > >> what is happening now.
> > >> 
> > >> Plan B: We keep a frozen version of the archives up to the
> > >> migration date under the pipermail directory and have the new
> > >> archives created in the archives directory.
> > >> 
> > >> All the pipermail archives have been restored as they were. The
> > >> new archives receives in their index a new link to point people
> > >> to the pipermails archive if looking for older archives.
> > >> 
> > >> this has been done a couple of hours ago (before Steve emails)
> > >> during a window, of ~ 10 hours, the mailing lists which slow
> > >> traffic aka the one which didn't received any email since the
> > >> migration ended up with an empty "archives" directory (aka a
> > >> 404), a file with explanation and redirection to pipermail has
> > >> been installed there.
> > >> 
> > >> Some work is still needed for the mailing lists which has been
> > >> transformed as readonly, this will be done in the next couple of
> > >> days  
> > > 
> > > It is too bad that a reference to a "no examples yet"
> > > month, such as, (at the time I write this):
> > > 
> > > https://lists.freebsd.org/archives/freebsd-toolchain/2021-June/date.html
> > > 
> > > does not show at least (Date view specific example):
> > > 
> > >   • Other periods:[ Previous, Date view ] [ List of Folders
> > > ]
> > >   • Other: [ Actives mailing lists ] [ Archives from
> > > mailman's time ]
> > > 
> > > when there are prior months available in
> > > https://lists.freebsd.org/archives/ or show at least just:
> > > 
> > >   • Other: [ Actives mailing lists ] [ Archives from
> > > mailman's time ]
> > > 
> > > when no prior months are available there.
> > >   
> > 
> > Looks like there are missing months.
> > 
> > Using freebsd-hackers as an example:
> > 
> > https://lists.freebsd.org/archives/freebsd-hackers/index.html
> > shows the oldest month being "May 2021".
> > 
> > But . . .
> > 
> > https://lists.freebsd.org/pipermail/freebsd-hackers/
> > shows the most recent month being "September 2020".
> > 
> > So: 2020-Oct through 2021-Apr are completely missing.
> > 
> > Then, going for more detail for 2020-Sep and 2021-May . . .
> > 
> > https://lists.freebsd.org/archives/freebsd-hackers/2021-May/date.html
> > shows "Starting Tue May 18 2021 - 21:07:44 UTC".
> > 
> > https://lists.freebsd.org/pipermail/freebsd-hackers/2020-September/date.html
> > shows "Ending: Tue Sep 15 14:12:20 UTC 2020".
> > 
> > So there are about 2 more half-months missing.
> > 
> > Some other lists have other date ranges, some similar,
> > some not.  
> 
> This missing month are being populated right now

Will historic links still work though (so that old references work and
current google results are ok)?

Just stumbled over this one:
https://lists.freebsd.org/archives/freebsd-jail/2017-March/003360.html

Cheers
Michael

> 
> Best regards,
> Bapt



-- 
Michael Gmelin

Re: Files in /etc containing empty VCSId header

[Mark Linimon]

On Mon, Jun 07, 2021 at 01:58:01PM -0600, Ian Lepore wrote:
> Sometimes it's a real interesting exercise to figure out where a
> file on your runtime system comes from in the source world.

A tangential problem I trip over is "what is on this SD card?"

It generally takes me 5-10 minutes to remember:

  strings //boot/kernel/kernel | tail

AFIAK it's the only way to be sure; nothing in //* seems
to have that data.

(Yes, in theory they all live in their own little box each of
which is labeled but things happen ...)

mcl

Re: kernel panic while copying files

[Hans Petter Selasky]

On 6/8/21 11:04 AM, Mateusz Guzik wrote:

Apparently caused by recent changes to CAM.

Let me know if you want more information.


Maybe you can print the *ccb being freed and figure out which device it 
belongs to.


--HPS

Re: kernel panic while copying files

[Mateusz Guzik]

Given how easy it is to reproduce perhaps you can spend a little bit
of time narrowing it down to a specific commit. You can do it with
git-bisect.

On 6/8/21, Gary Jennejohn  wrote:
> On Mon, 7 Jun 2021 16:54:11 -0400
> Mark Johnston  wrote:
>
>> On Mon, Jun 07, 2021 at 11:01:09AM +0200, Gary Jennejohn wrote:
>> > I've seen this panic three times in the last two days:
>> >
>> > [first panic]
>> > Unread portion of the kernel message buffer:
>> >
>> >
>> > Fatal trap 12: page fault while in kernel mode
>> > cpuid = 3; apic id = 03
>> > fault virtual address   = 0x801118000
>> > fault code  = supervisor write data, page not present
>> > instruction pointer = 0x20:0x808d2212
>> > stack pointer   = 0x28:0xfe00dbc8c760
>> > frame pointer   = 0x28:0xfe00dbc8c7a0
>> > code segment= base 0x0, limit 0xf, type 0x1b
>> > = DPL 0, pres 1, long 1, def32 0, gran 1
>> > processor eflags= interrupt enabled, resume, IOPL = 0
>> > current process = 28 (dom0)
>> > trap number = 12
>> > panic: page fault
>> > cpuid = 3
>> > time = 1622963058
>> > KDB: stack backtrace:
>> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>> > 0xfe00dbc8c410
>> > vpanic() at vpanic+0x181/frame 0xfe00dbc8c460
>> > panic() at panic+0x43/frame 0xfe00dbc8c4c0
>> > trap_fatal() at trap_fatal+0x387/frame 0xfe00dbc8c520
>> > trap_pfault() at trap_pfault+0x4f/frame 0xfe00dbc8c580
>> > trap() at trap+0x253/frame 0xfe00dbc8c690
>> > calltrap() at calltrap+0x8/frame 0xfe00dbc8c690
>> > --- trap 0xc, rip = 0x808d2212, rsp = 0xfe00dbc8c760, rbp =
>> > 0xfe00dbc8c7a0 ---
>> > zone_release() at zone_release+0x1f2/frame 0xfe00dbc8c7a0
>> > bucket_drain() at bucket_drain+0xda/frame 0xfe00dbc8c7d0
>> > bucket_cache_reclaim_domain() at bucket_cache_reclaim_domain+0x30a/frame
>> > 0xfe00dbc8c830
>> > zone_reclaim() at zone_reclaim+0x162/frame 0xfe00dbc8c880
>> > uma_reclaim_domain() at uma_reclaim_domain+0xa2/frame
>> > 0xfe00dbc8c8b0
>> > vm_pageout_worker() at vm_pageout_worker+0x41e/frame 0xfe00dbc8cb70
>> > vm_pageout() at vm_pageout+0x21e/frame 0xfe00dbc8cbb0
>> > fork_exit() at fork_exit+0x7e/frame 0xfe00dbc8cbf0
>> > fork_trampoline() at fork_trampoline+0xe/frame 0xfe00dbc8cbf0
>> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
>> > KDB: enter: panic
>> >
>> > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
>> > 55   __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
>> > pcpu,
>> >   pc_curthread)));
>> >
>> > One difference was that in the second and third panics the fault
>> > virtual
>> > address was 0x0.  But the backtrace was the same.
>> >
>> > Relevant info from the info.x files:
>> > Architecture: amd64
>> > Architecture Version: 2
>> > Version String: FreeBSD 14.0-CURRENT #33 main-n247184-1970d693039: Sat
>> > Jun
>> > 5 09:58:55 CEST 2021
>> >
>> > CPU: AMD Ryzen 5 1600 Six-Core Processor (3194.09-MHz
>> > K8-class CPU)
>> >   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1
>> > Stepping=1
>> >   AMD Features=0x2e500800
>> >   AMD
>> > Features2=0x35c233ff
>> >   AMD Extended Feature Extensions ID
>> > EBX=0x1007
>> >
>> > I have 16GiB of memory in the box.
>> >
>> > The panic occurred while copying files from an internal SATA SSD to a
>> > SATA 8TB disk in an external USB3 docking station.  The panic seems to
>> > occur quite quickly, after only a few files have been copied.
>> >
>> > swap is on a different internal disk.
>> >
>> > I can poke around in the crash dumps with kgdb if anyone wants more
>> > information.
>>
>> Are you running with invariants configured in the kernel?  If not,
>> please try to reproduce this in a kernel with
>>
>> options INVARIANT_SUPPORT
>> options INVARIANTS
>>
>> configured.
>>
>> A stack trace with line numbers would also be helpful.
>
> Thanks for the hint.  After enabling INVARIANTS the kernel panics as
> soon I turn on the external USB3 disk.  No user disk access required.
>
> Version String: FreeBSD 14.0-CURRENT #34 main-n247239-f570a6723e1: Tue Jun
> 8 09:34:32 CEST 2021
>
> Here the kgdb backtrace:
>
> Unread portion of the kernel message buffer:
> panic: Duplicate free of 0xf800356b9000 from zone
> 0xfe00dcbdd800(da_ccb) slab 0xf800356b9fd8(0)
> cpuid = 8
> time = 1623140519
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfe00c5f398c0
> vpanic() at vpanic+0x181/frame 0xfe00c5f39910
> panic() at panic+0x43/frame 0xfe00c5f39970
> uma_dbg_free() at uma_dbg_free+0x1e1/frame 0xfe00c5f399b0
> uma_zfree_arg() at uma_zfree_arg+0x147/frame 0xfe00c5f39a00
> camperiphdone() at camperiphdone+0x1b7/frame 0xfe00c5f39b20
> xpt_done_process() at xpt_done_process+0x3dd/frame 0xfe00c5f39b60
> xpt_done_td() at xpt_done_td+0xf5/frame 0xfe00c5f39bb0
> fork_exit() at fork_exit+0x80/frame 

Re: kernel panic while copying files

[Gary Jennejohn]

On Mon, 7 Jun 2021 16:54:11 -0400
Mark Johnston  wrote:

> On Mon, Jun 07, 2021 at 11:01:09AM +0200, Gary Jennejohn wrote:
> > I've seen this panic three times in the last two days:
> > 
> > [first panic]
> > Unread portion of the kernel message buffer:
> > 
> > 
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 3; apic id = 03
> > fault virtual address   = 0x801118000
> > fault code  = supervisor write data, page not present
> > instruction pointer = 0x20:0x808d2212
> > stack pointer   = 0x28:0xfe00dbc8c760
> > frame pointer   = 0x28:0xfe00dbc8c7a0
> > code segment= base 0x0, limit 0xf, type 0x1b
> > = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags= interrupt enabled, resume, IOPL = 0
> > current process = 28 (dom0)
> > trap number = 12
> > panic: page fault
> > cpuid = 3
> > time = 1622963058
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
> > 0xfe00dbc8c410
> > vpanic() at vpanic+0x181/frame 0xfe00dbc8c460
> > panic() at panic+0x43/frame 0xfe00dbc8c4c0
> > trap_fatal() at trap_fatal+0x387/frame 0xfe00dbc8c520
> > trap_pfault() at trap_pfault+0x4f/frame 0xfe00dbc8c580
> > trap() at trap+0x253/frame 0xfe00dbc8c690
> > calltrap() at calltrap+0x8/frame 0xfe00dbc8c690
> > --- trap 0xc, rip = 0x808d2212, rsp = 0xfe00dbc8c760, rbp = 
> > 0xfe00dbc8c7a0 ---
> > zone_release() at zone_release+0x1f2/frame 0xfe00dbc8c7a0
> > bucket_drain() at bucket_drain+0xda/frame 0xfe00dbc8c7d0
> > bucket_cache_reclaim_domain() at bucket_cache_reclaim_domain+0x30a/frame 
> > 0xfe00dbc8c830
> > zone_reclaim() at zone_reclaim+0x162/frame 0xfe00dbc8c880
> > uma_reclaim_domain() at uma_reclaim_domain+0xa2/frame 0xfe00dbc8c8b0
> > vm_pageout_worker() at vm_pageout_worker+0x41e/frame 0xfe00dbc8cb70
> > vm_pageout() at vm_pageout+0x21e/frame 0xfe00dbc8cbb0
> > fork_exit() at fork_exit+0x7e/frame 0xfe00dbc8cbf0
> > fork_trampoline() at fork_trampoline+0xe/frame 0xfe00dbc8cbf0
> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
> > KDB: enter: panic
> > 
> > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> > 55   __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu,
> >pc_curthread)));
> > 
> > One difference was that in the second and third panics the fault virtual
> > address was 0x0.  But the backtrace was the same.
> > 
> > Relevant info from the info.x files:
> > Architecture: amd64
> > Architecture Version: 2
> > Version String: FreeBSD 14.0-CURRENT #33 main-n247184-1970d693039: Sat Jun
> > 5 09:58:55 CEST 2021
> > 
> > CPU: AMD Ryzen 5 1600 Six-Core Processor (3194.09-MHz K8-class 
> > CPU)
> >   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1  Stepping=1
> >   AMD Features=0x2e500800
> >   AMD 
> > Features2=0x35c233ff
> >   AMD Extended Feature Extensions ID 
> > EBX=0x1007
> > 
> > I have 16GiB of memory in the box.
> > 
> > The panic occurred while copying files from an internal SATA SSD to a
> > SATA 8TB disk in an external USB3 docking station.  The panic seems to
> > occur quite quickly, after only a few files have been copied.
> > 
> > swap is on a different internal disk.
> > 
> > I can poke around in the crash dumps with kgdb if anyone wants more
> > information.  
> 
> Are you running with invariants configured in the kernel?  If not,
> please try to reproduce this in a kernel with
> 
> options INVARIANT_SUPPORT
> options INVARIANTS
> 
> configured.
> 
> A stack trace with line numbers would also be helpful.

Thanks for the hint.  After enabling INVARIANTS the kernel panics as
soon I turn on the external USB3 disk.  No user disk access required.

Version String: FreeBSD 14.0-CURRENT #34 main-n247239-f570a6723e1: Tue Jun
8 09:34:32 CEST 2021

Here the kgdb backtrace:

Unread portion of the kernel message buffer:
panic: Duplicate free of 0xf800356b9000 from zone 
0xfe00dcbdd800(da_ccb) slab 0xf800356b9fd8(0)
cpuid = 8
time = 1623140519
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe00c5f398c0
vpanic() at vpanic+0x181/frame 0xfe00c5f39910
panic() at panic+0x43/frame 0xfe00c5f39970
uma_dbg_free() at uma_dbg_free+0x1e1/frame 0xfe00c5f399b0
uma_zfree_arg() at uma_zfree_arg+0x147/frame 0xfe00c5f39a00
camperiphdone() at camperiphdone+0x1b7/frame 0xfe00c5f39b20
xpt_done_process() at xpt_done_process+0x3dd/frame 0xfe00c5f39b60
xpt_done_td() at xpt_done_td+0xf5/frame 0xfe00c5f39bb0
fork_exit() at fork_exit+0x80/frame 0xfe00c5f39bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfe00c5f39bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic

__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
55  __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct 
pcpu,
(kgdb) bt
#0  __curthread () at