GPU firmware: could not load firmware image, error 2

[Graham Perrin]

On 20/08/2023 23:06, Graham Perrin wrote:


Some relevant messages at.

Should I be concerned by error 2 in these contexts?

% pkg iinfo drm-510-kmod
drm-510-kmod-5.10.163_7
% uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-ALPHA2 FreeBSD 14.0-ALPHA2 amd64 
1400094 #4 main-n264868-edacf4b4824a-dirty: Fri Aug 18 23:46:09 BST 
2023grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
  amd64 1400094 1400094


Off-list, someone wrote:


I believe you have to install drm-515-kmod


It's not a requirement.

I did try it (and encountered the errors) before reverting to drm-510-kmod.

OT: These errors aside, drm-515-kmod is, for me, so unreliable that I 
can't use it; the bugs are in GitHub.

GPU firmware: could not load firmware image, error 2

[Graham Perrin]

Some relevant messages at.

Should I be concerned by error 2 in these contexts?

% pkg iinfo drm-510-kmod
drm-510-kmod-5.10.163_7
% uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-ALPHA2 FreeBSD 14.0-ALPHA2 amd64 
1400094 #4 main-n264868-edacf4b4824a-dirty: Fri Aug 18 23:46:09 BST 
2023grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
  amd64 1400094 1400094

Boot failures with 14.0-ALPHA1 and 14.0-ALPHA2

[Graham Perrin]

On 18/08/2023 13:21, Graham Perrin wrote:

I'll update to ALPHA2.


Another failure.

In this case, removal of a USB device was unnecessary; the next start 
succeeded.


% bectl list -c creation | grep edacf4b4824a
n264868-edacf4b4824a-a NR /  460G  2023-08-19 03:54
n264868-edacf4b4824a-b -  -  52.1M 2023-08-20 20:20
% uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-ALPHA2 FreeBSD 14.0-ALPHA2 amd64 
1400094 #4 main-n264868-edacf4b4824a-dirty: Fri Aug 18 23:46:09 BST 2023 
grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.a

md64/sys/GENERIC amd64 1400094 1400094
%


In the meantime, briefly: 

– no visible progress beyond the four lines of EFI framebuffer 
information.


In each case, I worked around by removing a USB flash drive that's 
used for L2ARC.


The most recent incident was preceded by a power button shutdown, in 
response to a drm-510-kmod blackout (GPU lockup etc.) whilst typing in 
Firefox (comparable to 
 for drm-515-kmod 
5.15.25).


The prior incident was also preceded by a power button shutdown (in 
response to a blackout at Plasma log out time). This was remarkable in 
that there was an unexpected restart, when I expected the computer to 
power off.

Re: Speed improvements in ZFS

[Konstantin Belousov]

On Sun, Aug 20, 2023 at 11:07:08PM +0200, Mateusz Guzik wrote:
> On 8/20/23, Alexander Leidinger  wrote:
> > Am 2023-08-20 22:02, schrieb Mateusz Guzik:
> >> On 8/20/23, Alexander Leidinger  wrote:
> >>> Am 2023-08-20 19:10, schrieb Mateusz Guzik:
>  On 8/18/23, Alexander Leidinger  wrote:
> >>>
> > I have a 51MB text file, compressed to about 1MB. Are you interested
> > to
> > get it?
> >
> 
>  Your problem is not the vnode limit, but nullfs.
> 
>  https://people.freebsd.org/~mjg/netchild-periodic-find.svg
> >>>
> >>> 122 nullfs mounts on this system. And every jail I setup has several
> >>> null mounts. One basesystem mounted into every jail, and then shared
> >>> ports (packages/distfiles/ccache) across all of them.
> >>>
>  First, some of the contention is notorious VI_LOCK in order to do
>  anything.
> 
>  But more importantly the mind-boggling off-cpu time comes from
>  exclusive locking which should not be there to begin with -- as in
>  that xlock in stat should be a slock.
> 
>  Maybe I'm going to look into it later.
> >>>
> >>> That would be fantastic.
> >>>
> >>
> >> I did a quick test, things are shared locked as expected.
> >>
> >> However, I found the following:
> >> if ((xmp->nullm_flags & NULLM_CACHE) != 0) {
> >> mp->mnt_kern_flag |=
> >> lowerrootvp->v_mount->mnt_kern_flag &
> >> (MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED |
> >> MNTK_EXTENDED_SHARED);
> >> }
> >>
> >> are you using the "nocache" option? it has a side effect of xlocking
> >
> > I use noatime, noexec, nosuid, nfsv4acls. I do NOT use nocache.
> >
> 
> If you don't have "nocache" on null mounts, then I don't see how this
> could happen.

There is also MNTK_NULL_NOCACHE on lower fs, which is currently set for
fuse and nfs at least.

Re: Speed improvements in ZFS

[Mateusz Guzik]

On 8/20/23, Alexander Leidinger  wrote:
> Am 2023-08-20 22:02, schrieb Mateusz Guzik:
>> On 8/20/23, Alexander Leidinger  wrote:
>>> Am 2023-08-20 19:10, schrieb Mateusz Guzik:
 On 8/18/23, Alexander Leidinger  wrote:
>>>
> I have a 51MB text file, compressed to about 1MB. Are you interested
> to
> get it?
>

 Your problem is not the vnode limit, but nullfs.

 https://people.freebsd.org/~mjg/netchild-periodic-find.svg
>>>
>>> 122 nullfs mounts on this system. And every jail I setup has several
>>> null mounts. One basesystem mounted into every jail, and then shared
>>> ports (packages/distfiles/ccache) across all of them.
>>>
 First, some of the contention is notorious VI_LOCK in order to do
 anything.

 But more importantly the mind-boggling off-cpu time comes from
 exclusive locking which should not be there to begin with -- as in
 that xlock in stat should be a slock.

 Maybe I'm going to look into it later.
>>>
>>> That would be fantastic.
>>>
>>
>> I did a quick test, things are shared locked as expected.
>>
>> However, I found the following:
>> if ((xmp->nullm_flags & NULLM_CACHE) != 0) {
>> mp->mnt_kern_flag |=
>> lowerrootvp->v_mount->mnt_kern_flag &
>> (MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED |
>> MNTK_EXTENDED_SHARED);
>> }
>>
>> are you using the "nocache" option? it has a side effect of xlocking
>
> I use noatime, noexec, nosuid, nfsv4acls. I do NOT use nocache.
>

If you don't have "nocache" on null mounts, then I don't see how this
could happen.

-- 
Mateusz Guzik 

Re: Speed improvements in ZFS

[Alexander Leidinger]

Am 2023-08-20 22:02, schrieb Mateusz Guzik:

On 8/20/23, Alexander Leidinger  wrote:

Am 2023-08-20 19:10, schrieb Mateusz Guzik:

On 8/18/23, Alexander Leidinger  wrote:



I have a 51MB text file, compressed to about 1MB. Are you interested
to
get it?



Your problem is not the vnode limit, but nullfs.

https://people.freebsd.org/~mjg/netchild-periodic-find.svg


122 nullfs mounts on this system. And every jail I setup has several
null mounts. One basesystem mounted into every jail, and then shared
ports (packages/distfiles/ccache) across all of them.


First, some of the contention is notorious VI_LOCK in order to do
anything.

But more importantly the mind-boggling off-cpu time comes from
exclusive locking which should not be there to begin with -- as in
that xlock in stat should be a slock.

Maybe I'm going to look into it later.


That would be fantastic.



I did a quick test, things are shared locked as expected.

However, I found the following:
if ((xmp->nullm_flags & NULLM_CACHE) != 0) {
mp->mnt_kern_flag |= 
lowerrootvp->v_mount->mnt_kern_flag &

(MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED |
MNTK_EXTENDED_SHARED);
}

are you using the "nocache" option? it has a side effect of xlocking


I use noatime, noexec, nosuid, nfsv4acls. I do NOT use nocache.

Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

Re: Speed improvements in ZFS

[Mateusz Guzik]

On 8/20/23, Alexander Leidinger  wrote:
> Am 2023-08-20 19:10, schrieb Mateusz Guzik:
>> On 8/18/23, Alexander Leidinger  wrote:
>
>>> I have a 51MB text file, compressed to about 1MB. Are you interested
>>> to
>>> get it?
>>>
>>
>> Your problem is not the vnode limit, but nullfs.
>>
>> https://people.freebsd.org/~mjg/netchild-periodic-find.svg
>
> 122 nullfs mounts on this system. And every jail I setup has several
> null mounts. One basesystem mounted into every jail, and then shared
> ports (packages/distfiles/ccache) across all of them.
>
>> First, some of the contention is notorious VI_LOCK in order to do
>> anything.
>>
>> But more importantly the mind-boggling off-cpu time comes from
>> exclusive locking which should not be there to begin with -- as in
>> that xlock in stat should be a slock.
>>
>> Maybe I'm going to look into it later.
>
> That would be fantastic.
>

I did a quick test, things are shared locked as expected.

However, I found the following:
if ((xmp->nullm_flags & NULLM_CACHE) != 0) {
mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag &
(MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED |
MNTK_EXTENDED_SHARED);
}

are you using the "nocache" option? it has a side effect of xlocking

-- 
Mateusz Guzik 

Re: Speed improvements in ZFS

[Alexander Leidinger]

Am 2023-08-20 19:10, schrieb Mateusz Guzik:

On 8/18/23, Alexander Leidinger  wrote:


I have a 51MB text file, compressed to about 1MB. Are you interested 
to

get it?



Your problem is not the vnode limit, but nullfs.

https://people.freebsd.org/~mjg/netchild-periodic-find.svg


122 nullfs mounts on this system. And every jail I setup has several 
null mounts. One basesystem mounted into every jail, and then shared 
ports (packages/distfiles/ccache) across all of them.


First, some of the contention is notorious VI_LOCK in order to do 
anything.


But more importantly the mind-boggling off-cpu time comes from
exclusive locking which should not be there to begin with -- as in
that xlock in stat should be a slock.

Maybe I'm going to look into it later.


That would be fantastic.

Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

Re: 100% CPU time for sysctl command, not killable

[Alexander Leidinger]

Am 2023-08-20 18:55, schrieb Mina Galić:


procstat(1) kstack could be helpful here.

 Original Message 
On 20 Aug 2023, 17:29, Alexander Leidinger alexan...@leidinger.net> 
wrote:


Hi, sysctl kern.maxvnodes=1048576000 results in 100% CPU and a 
non-killable sysctl program. This is somewhat unexpected... Bye, 
Alexander. -- http://www.Leidinger.net alexan...@leidinger.net: PGP 
0x8F31830F9F2772BF http://www.FreeBSD.org netch...@freebsd.org : PGP 
0x8F31830F9F2772BF


  PIDTID COMMTDNAME  KSTACK
94391 118678 sysctl  -   sysctl_maxvnodes 
sysctl_root_handler_locked sysctl_root userland_sysctl sys___sysctl 
amd64_syscall fast_syscall_common


Bye,

Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

Re: Speed improvements in ZFS

[Mateusz Guzik]

On 8/18/23, Alexander Leidinger  wrote:
> Am 2023-08-16 18:48, schrieb Alexander Leidinger:
>> Am 2023-08-15 23:29, schrieb Mateusz Guzik:
>>> On 8/15/23, Alexander Leidinger  wrote:
 Am 2023-08-15 14:41, schrieb Mateusz Guzik:

> With this in mind can you provide: sysctl kern.maxvnodes
> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes
> vfs.recycles_free vfs.recycles

 After a reboot:
 kern.maxvnodes: 10485760
 vfs.wantfreevnodes: 2621440
 vfs.freevnodes: 24696
 vfs.vnodes_created: 1658162
 vfs.numvnodes: 173937
 vfs.recycles_free: 0
 vfs.recycles: 0
>>
>> New values after one rund of periodic:
>> kern.maxvnodes: 10485760
>> vfs.wantfreevnodes: 2621440
>> vfs.freevnodes: 356202
>> vfs.vnodes_created: 427696288
>> vfs.numvnodes: 532620
>> vfs.recycles_free: 20213257
>> vfs.recycles: 0
>
> And after the second round which only took 7h this night:
> kern.maxvnodes: 10485760
> vfs.wantfreevnodes: 2621440
> vfs.freevnodes: 3071754
> vfs.vnodes_created: 1275963316
> vfs.numvnodes: 3414906
> vfs.recycles_free: 58411371
> vfs.recycles: 0
>
> Meanwhile if there is tons of recycles, you can damage control by
> bumping kern.maxvnodes.
>>
>> What's the difference between recycles and recycles_free? Does the
>> above count as bumping the maxvnodes?
>
> ^
>
 Looks like there are not much free directly after the reboot. I will
 check the values tomorrow after the periodic run again and maybe
 increase by 10 or 100 so see if it makes a difference.

> If this is not the problem you can use dtrace to figure it out.

 dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or
 something else?

>>>
>>> I mean checking where find is spending time instead of speculating.
>>>
>>> There is no productized way to do it so to speak, but the following
>>> crapper should be good enough:
>> [script]
>>
>> I will let it run this night.
>
> I have a 51MB text file, compressed to about 1MB. Are you interested to
> get it?
>

Your problem is not the vnode limit, but nullfs.

https://people.freebsd.org/~mjg/netchild-periodic-find.svg

First, some of the contention is notorious VI_LOCK in order to do anything.

But more importantly the mind-boggling off-cpu time comes from
exclusive locking which should not be there to begin with -- as in
that xlock in stat should be a slock.

Maybe I'm going to look into it later.

-- 
Mateusz Guzik 

Re: 100% CPU time for sysctl command, not killable

[Mina Galić]

procstat(1) kstack could be helpful here.

 Original Message 
On 20 Aug 2023, 17:29, Alexander Leidinger wrote:

> Hi, sysctl kern.maxvnodes=1048576000 results in 100% CPU and a non-killable 
> sysctl program. This is somewhat unexpected... Bye, Alexander. -- 
> http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF 
> http://www.FreeBSD.org netch...@freebsd.org : PGP 0x8F31830F9F2772BF

Re: 100% CPU time for sysctl command, not killable

[Ivan Quitschal]

Sounds  like a fork bomb discovered

Obter o Outlook para Android

From: owner-freebsd-curr...@freebsd.org  on 
behalf of Alexander Leidinger 
Sent: Sunday, August 20, 2023 1:29:40 PM
To: curr...@freebsd.org 
Subject: 100% CPU time for sysctl command, not killable

Hi,

sysctl kern.maxvnodes=1048576000 results in 100% CPU and a non-killable
sysctl program. This is somewhat unexpected...

Bye,
Alexander.

--
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.leidinger.net%2F=05%7C01%7C%7C8154f1ba1c56451df44f08dba19ab6fb%7C84df9e7fe9f640afb435%7C1%7C0%7C638281458099404081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=1w9Qc6PlRiJU%2FYrGjSiyLGIbWfoekhX2XbRE%2FMr2MB8%3D=0
 alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freebsd.org%2F=05%7C01%7C%7C8154f1ba1c56451df44f08dba19ab6fb%7C84df9e7fe9f640afb435%7C1%7C0%7C638281458099404081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=XONqhX5V5duH8oyRVwnAiruzHzcnIGCcjbWVDLscILU%3D=0
netch...@freebsd.org  : PGP 0x8F31830F9F2772BF

100% CPU time for sysctl command, not killable

[Alexander Leidinger]

Hi,

sysctl kern.maxvnodes=1048576000 results in 100% CPU and a non-killable 
sysctl program. This is somewhat unexpected...


Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF

Re: ZFS deadlock in 14

[Mark Millard]

Dag-Erling_Smørgrav  wrote on
Date: Sun, 20 Aug 2023 13:00:27 UTC :

> Alexander Motin  writes:
> > Unfortunately I think the current code in main should still suffer
> > from this specific deadlock. cd25b0f740 fixes some deadlocks in this
> > area, may be that is why you are getting issues less often, but I
> > don't believe it fixes this specific one, may be you was just lucky.
> 
> Yep, it took a while but I got a deadlock with unmodified 4f14d4b6b7.

Are any of you testing based on USE_TMPFS=no and such, so that
ZFS handles all file system activity, while allowing multiple
builders to be active as well? In my case, I also have:
ALLOW_MAKE_JOBS=yes

In my testing, pkg builds and then the other builders report
"Builder starting" but end up stuck waiting in a loop with
timeouts involved ("vlruwk") before ever reporting any more.
Some progress is made in the original builder before I
eventually stop poudriere-devel.

Am I the only one that gets such behavior for this kind of
context?

===
Mark Millard
marklmi at yahoo.com