Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Sun, 03 Sep 2023 15:17:36 +0200 "Herbert J. Skuhra" wrote: [SNIP] > Probably best to file a PR: https://bugs.freebsd.org/bugzilla/ > Bugzilla 273543 -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Sat, 02 Sep 2023 18:02:03 +0200, Gary Jennejohn wrote: > > On Sat, 02 Sep 2023 15:36:36 +0200 > "Herbert J. Skuhra" wrote: > > > On Fri, 01 Sep 2023 18:05:34 +0200, Gary Jennejohn wrote: > > > > > > On Fri, 1 Sep 2023 14:43:21 + > > > Gary Jennejohn wrote: > > > > > > > A git-bisect is probably required. > > > > > > > > > > I did a bisect and the result was commit > > > 9a7add6d01f3c5f7eba811e794cf860d2bce131d. > > > > > > However, that can't be correct because this commit was made on > > > Mon Jul 17 19:29:20 2023 and my FBSD-14 kernel from August 13th > > > boots successfully :( > > > > Commit date is August 19th, 2023(!): > > > > commit 9a7add6d01f3c5f7eba811e794cf860d2bce131d > > Author: Colin Percival > > AuthorDate: Mon Jul 17 19:29:20 2023 -0700 > > Commit: Colin Percival > > CommitDate: Sat Aug 19 22:04:56 2023 -0700 > > > > > > Reverting this commit seems to resolve the issue for me: > > > > FreeBSD 15.0-CURRENT amd64 150 #0 main-n265137-2ad756a6bbb3 > > > > $ git status > > On branch main > > Your branch is up to date with 'freebsd/main'. > > > > You are currently reverting commit 9a7add6d01f3. > > (all conflicts fixed: run "git revert --continue") > > (use "git revert --skip" to skip this patch) > > (use "git revert --abort" to cancel the revert operation) > > > > Changes to be committed: > > (use "git restore --staged ..." to unstage) > > modified: sys/kern/init_main.c > > > > # dmesg |egrep "(amdsmn|amdtemp)" > > amdsmn0: on hostb0 > > amdtemp0: on hostb0 > > > > $ sysctl kern.conftxt |grep amdt > > device amdtemp > > > > Really? I did a git log and July 17 is what pops out for this commit. > > Ah, I see that git log doesn't show the commit date. > > So I guess that the git bisect really did find the commit which caused > all our problems. > > If reverting it fixes things then this requires some action from Colin > Percival. > > This would also explain why my FBSD-14 kernel from August 13 was > OK. Probably best to file a PR: https://bugs.freebsd.org/bugzilla/ -- Herbert
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Sat, 02 Sep 2023 15:36:36 +0200 "Herbert J. Skuhra" wrote: > On Fri, 01 Sep 2023 18:05:34 +0200, Gary Jennejohn wrote: > > > > On Fri, 1 Sep 2023 14:43:21 + > > Gary Jennejohn wrote: > > > > > A git-bisect is probably required. > > > > > > > I did a bisect and the result was commit > > 9a7add6d01f3c5f7eba811e794cf860d2bce131d. > > > > However, that can't be correct because this commit was made on > > Mon Jul 17 19:29:20 2023 and my FBSD-14 kernel from August 13th > > boots successfully :( > > Commit date is August 19th, 2023(!): > > commit 9a7add6d01f3c5f7eba811e794cf860d2bce131d > Author: Colin Percival > AuthorDate: Mon Jul 17 19:29:20 2023 -0700 > Commit: Colin Percival > CommitDate: Sat Aug 19 22:04:56 2023 -0700 > > > Reverting this commit seems to resolve the issue for me: > > FreeBSD 15.0-CURRENT amd64 150 #0 main-n265137-2ad756a6bbb3 > > $ git status > On branch main > Your branch is up to date with 'freebsd/main'. > > You are currently reverting commit 9a7add6d01f3. > (all conflicts fixed: run "git revert --continue") > (use "git revert --skip" to skip this patch) > (use "git revert --abort" to cancel the revert operation) > > Changes to be committed: > (use "git restore --staged ..." to unstage) > modified: sys/kern/init_main.c > > # dmesg |egrep "(amdsmn|amdtemp)" > amdsmn0: on hostb0 > amdtemp0: on hostb0 > > $ sysctl kern.conftxt |grep amdt > device amdtemp > Really? I did a git log and July 17 is what pops out for this commit. Ah, I see that git log doesn't show the commit date. So I guess that the git bisect really did find the commit which caused all our problems. If reverting it fixes things then this requires some action from Colin Percival. This would also explain why my FBSD-14 kernel from August 13 was OK. -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 01 Sep 2023 18:05:34 +0200, Gary Jennejohn wrote: > > On Fri, 1 Sep 2023 14:43:21 + > Gary Jennejohn wrote: > > > A git-bisect is probably required. > > > > I did a bisect and the result was commit > 9a7add6d01f3c5f7eba811e794cf860d2bce131d. > > However, that can't be correct because this commit was made on > Mon Jul 17 19:29:20 2023 and my FBSD-14 kernel from August 13th > boots successfully :( Commit date is August 19th, 2023(!): commit 9a7add6d01f3c5f7eba811e794cf860d2bce131d Author: Colin Percival AuthorDate: Mon Jul 17 19:29:20 2023 -0700 Commit: Colin Percival CommitDate: Sat Aug 19 22:04:56 2023 -0700 Reverting this commit seems to resolve the issue for me: FreeBSD 15.0-CURRENT amd64 150 #0 main-n265137-2ad756a6bbb3 $ git status On branch main Your branch is up to date with 'freebsd/main'. You are currently reverting commit 9a7add6d01f3. (all conflicts fixed: run "git revert --continue") (use "git revert --skip" to skip this patch) (use "git revert --abort" to cancel the revert operation) Changes to be committed: (use "git restore --staged ..." to unstage) modified: sys/kern/init_main.c # dmesg |egrep "(amdsmn|amdtemp)" amdsmn0: on hostb0 amdtemp0: on hostb0 $ sysctl kern.conftxt |grep amdt device amdtemp -- Herbert
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 1 Sep 2023 16:00:03 -0600 Warner Losh wrote: > I think that the problem is that admsmn has probed, but not attached (or > failed to attach for some reason), so we find the device, but it's not > initialized yet, so when we call amdsmn_read, it tries to lock a mutex > that's not yet initialized. > > Not sure why this is happening, or why loading it as modules fixes it... > > But since I don't have the hardware, I can't help more. Sorry. > Might it be worth adding an entry to /usr/src/UPDATING for users with older Zen versions like mine (Zen 1 and Zen 2), recommending kldload'ing of amdsmn and amdtemp? -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
I think that the problem is that admsmn has probed, but not attached (or failed to attach for some reason), so we find the device, but it's not initialized yet, so when we call amdsmn_read, it tries to lock a mutex that's not yet initialized. Not sure why this is happening, or why loading it as modules fixes it... But since I don't have the hardware, I can't help more. Sorry. Warner On Fri, Sep 1, 2023 at 10:21 AM Gary Jennejohn wrote: > On Fri, 01 Sep 2023 17:14:02 +0200 > "Herbert J. Skuhra" wrote: > > > On Fri, 01 Sep 2023 16:04:41 +0200, Gary Jennejohn wrote: > > > > > > On Fri, 01 Sep 2023 14:15:20 +0200 > > > "Herbert J. Skuhra" wrote: > > > > > > > On Fri, 01 Sep 2023 13:03:14 +0200, Gary Jennejohn wrote: > > > > > > > > > > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 > 3700X. > > > > > > > > > > These are respectively Zen 1 and Zen 2 CPUs. > > > > > > > > > > I built a kernel on both computers using the FreeBSD-15 source > tree. > > > > > > > > > > If I include the amdtemp device in my kernel file BOTH computers > end up > > > > > with a kernel panic while trying to attach the amdtemp device. > > > > > > > > > > If I remove amdtemp both computers boot without any issues. > > > > > > > > > > I suspect that this commit is the cause: > > > > > > > > > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > > > > > Author: Akio Morita > > > > > Date: Tue Aug 1 22:32:12 2023 +0200 > > > > > > > > > > amdsmn(4), amdtemp(4): add support for Zen 4 > > > > > > > > > > Zen 4 support, tested on Ryzen 9 7900 > > > > > > > > > > Reviewed by:imp (previous version), mhorne > > > > > Approved by:mhorne > > > > > Obtained from: > http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > > > > > Differential Revision: https://reviews.freebsd.org/D41049 > > > > > > > > Thanks for sharing your findings. > > > > > > > > Now I probably know why my old kernel from stable/13 no longer booted > > > > after updating to stable/14. I've create a new kernel config and > > > > forgot to add "device amdtemp" & "device amdsmn" and forgot about the > > > > issue. After removing only "device amdtemp" from my old kernel config > > > > it boots again. > > > > > > > > Unfortunately reverting this commit (git revert -n 323a94afb623) > > > > doesn't resolve this issue. Old kernel does not boot if "device > > > > amdtemp" is enabled. Probably wrong commit or I am doing somethig > > > > wrong!? > > > > > > > > > > Strange. My FreeBSD-14 kernel boots with device amdtemp (which > automatically > > > results in amdsmn being included). It's FreeBSD-15 which fails for me. > > > > 1. 'kload amdtemp' works: > >121 0x81e7c000 3160 amdtemp.ko > >131 0x81e8 2138 amdsmn.ko > > > >amdsmn0: on hostb0 > >amdtemp0: on hostb0 > > > > 2. GENERIC boots fine. The following kernel does not: > > > >include GENERIC > > > >ident TEST > >device amdtemp > > > > 3. Unfortunately this is a remote server without a serial console. I > > don't get a crashdump and I can't find anything in /var/log/messages. > > > > 4. I have no good revision for stable/14 and main. On main I always > > use GENERIC-NODEBUG. :-( > > > > Thanks, Herbert! kldload'ing amdsmn and amdtemp really does work! > > Now I can run FBSD-15 :) > > -- > Gary Jennejohn > >
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 01 Sep 2023 17:14:02 +0200 "Herbert J. Skuhra" wrote: > On Fri, 01 Sep 2023 16:04:41 +0200, Gary Jennejohn wrote: > > > > On Fri, 01 Sep 2023 14:15:20 +0200 > > "Herbert J. Skuhra" wrote: > > > > > On Fri, 01 Sep 2023 13:03:14 +0200, Gary Jennejohn wrote: > > > > > > > > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 > > > > 3700X. > > > > > > > > These are respectively Zen 1 and Zen 2 CPUs. > > > > > > > > I built a kernel on both computers using the FreeBSD-15 source tree. > > > > > > > > If I include the amdtemp device in my kernel file BOTH computers end up > > > > with a kernel panic while trying to attach the amdtemp device. > > > > > > > > If I remove amdtemp both computers boot without any issues. > > > > > > > > I suspect that this commit is the cause: > > > > > > > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > > > > Author: Akio Morita > > > > Date: Tue Aug 1 22:32:12 2023 +0200 > > > > > > > > amdsmn(4), amdtemp(4): add support for Zen 4 > > > > > > > > Zen 4 support, tested on Ryzen 9 7900 > > > > > > > > Reviewed by:imp (previous version), mhorne > > > > Approved by:mhorne > > > > Obtained from: > > > > http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > > > > Differential Revision: https://reviews.freebsd.org/D41049 > > > > > > Thanks for sharing your findings. > > > > > > Now I probably know why my old kernel from stable/13 no longer booted > > > after updating to stable/14. I've create a new kernel config and > > > forgot to add "device amdtemp" & "device amdsmn" and forgot about the > > > issue. After removing only "device amdtemp" from my old kernel config > > > it boots again. > > > > > > Unfortunately reverting this commit (git revert -n 323a94afb623) > > > doesn't resolve this issue. Old kernel does not boot if "device > > > amdtemp" is enabled. Probably wrong commit or I am doing somethig > > > wrong!? > > > > > > > Strange. My FreeBSD-14 kernel boots with device amdtemp (which > > automatically > > results in amdsmn being included). It's FreeBSD-15 which fails for me. > > 1. 'kload amdtemp' works: >121 0x81e7c000 3160 amdtemp.ko >131 0x81e8 2138 amdsmn.ko > >amdsmn0: on hostb0 >amdtemp0: on hostb0 > > 2. GENERIC boots fine. The following kernel does not: > >include GENERIC > >ident TEST >device amdtemp > > 3. Unfortunately this is a remote server without a serial console. I > don't get a crashdump and I can't find anything in /var/log/messages. > > 4. I have no good revision for stable/14 and main. On main I always > use GENERIC-NODEBUG. :-( > Thanks, Herbert! kldload'ing amdsmn and amdtemp really does work! Now I can run FBSD-15 :) -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 1 Sep 2023 14:43:21 + Gary Jennejohn wrote: > A git-bisect is probably required. > I did a bisect and the result was commit 9a7add6d01f3c5f7eba811e794cf860d2bce131d. However, that can't be correct because this commit was made on Mon Jul 17 19:29:20 2023 and my FBSD-14 kernel from August 13th boots successfully :( However, "Herbert J. Skuhra" says that kldload'ing amdtemp works. So I'll give that a try. -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 01 Sep 2023 16:04:41 +0200, Gary Jennejohn wrote: > > On Fri, 01 Sep 2023 14:15:20 +0200 > "Herbert J. Skuhra" wrote: > > > On Fri, 01 Sep 2023 13:03:14 +0200, Gary Jennejohn wrote: > > > > > > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. > > > > > > These are respectively Zen 1 and Zen 2 CPUs. > > > > > > I built a kernel on both computers using the FreeBSD-15 source tree. > > > > > > If I include the amdtemp device in my kernel file BOTH computers end up > > > with a kernel panic while trying to attach the amdtemp device. > > > > > > If I remove amdtemp both computers boot without any issues. > > > > > > I suspect that this commit is the cause: > > > > > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > > > Author: Akio Morita > > > Date: Tue Aug 1 22:32:12 2023 +0200 > > > > > > amdsmn(4), amdtemp(4): add support for Zen 4 > > > > > > Zen 4 support, tested on Ryzen 9 7900 > > > > > > Reviewed by:imp (previous version), mhorne > > > Approved by:mhorne > > > Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > > > Differential Revision: https://reviews.freebsd.org/D41049 > > > > Thanks for sharing your findings. > > > > Now I probably know why my old kernel from stable/13 no longer booted > > after updating to stable/14. I've create a new kernel config and > > forgot to add "device amdtemp" & "device amdsmn" and forgot about the > > issue. After removing only "device amdtemp" from my old kernel config > > it boots again. > > > > Unfortunately reverting this commit (git revert -n 323a94afb623) > > doesn't resolve this issue. Old kernel does not boot if "device > > amdtemp" is enabled. Probably wrong commit or I am doing somethig > > wrong!? > > > > Strange. My FreeBSD-14 kernel boots with device amdtemp (which automatically > results in amdsmn being included). It's FreeBSD-15 which fails for me. 1. 'kload amdtemp' works: 121 0x81e7c000 3160 amdtemp.ko 131 0x81e8 2138 amdsmn.ko amdsmn0: on hostb0 amdtemp0: on hostb0 2. GENERIC boots fine. The following kernel does not: include GENERIC identTEST device amdtemp 3. Unfortunately this is a remote server without a serial console. I don't get a crashdump and I can't find anything in /var/log/messages. 4. I have no good revision for stable/14 and main. On main I always use GENERIC-NODEBUG. :-( -- Herbert
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 1 Sep 2023 14:23:36 + Gary Jennejohn wrote: > Now that I look at the date of my FreeBSD-14 kernel I see that it's from > August 13, so this commit is perhaps not the cause of my FreeBSD-15 > kernel panicking at boot time, since FBSD-14 boots OK. > > Nonetheless, amdtemp or maybe amdsmn seems to be related. > Since my FBSD-14 kernel is from August 13 and FBSD-15 appeared on August 24 there were 10 to 11 days of commits in between. That makes it much more difficult to pinpoint the cause. A git-bisect is probably required. -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 1 Sep 2023 11:03:14 + Gary Jennejohn wrote: > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. > > These are respectively Zen 1 and Zen 2 CPUs. > > I built a kernel on both computers using the FreeBSD-15 source tree. > > If I include the amdtemp device in my kernel file BOTH computers end up > with a kernel panic while trying to attach the amdtemp device. > > If I remove amdtemp both computers boot without any issues. > > I suspect that this commit is the cause: > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > Author: Akio Morita > Date: Tue Aug 1 22:32:12 2023 +0200 > > amdsmn(4), amdtemp(4): add support for Zen 4 > > Zen 4 support, tested on Ryzen 9 7900 > > Reviewed by:imp (previous version), mhorne > Approved by:mhorne > Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > Differential Revision: https://reviews.freebsd.org/D41049 > Now that I look at the date of my FreeBSD-14 kernel I see that it's from August 13, so this commit is perhaps not the cause of my FreeBSD-15 kernel panicking at boot time, since FBSD-14 boots OK. Nonetheless, amdtemp or maybe amdsmn seems to be related. -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 01 Sep 2023 14:15:20 +0200 "Herbert J. Skuhra" wrote: > On Fri, 01 Sep 2023 13:03:14 +0200, Gary Jennejohn wrote: > > > > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. > > > > These are respectively Zen 1 and Zen 2 CPUs. > > > > I built a kernel on both computers using the FreeBSD-15 source tree. > > > > If I include the amdtemp device in my kernel file BOTH computers end up > > with a kernel panic while trying to attach the amdtemp device. > > > > If I remove amdtemp both computers boot without any issues. > > > > I suspect that this commit is the cause: > > > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > > Author: Akio Morita > > Date: Tue Aug 1 22:32:12 2023 +0200 > > > > amdsmn(4), amdtemp(4): add support for Zen 4 > > > > Zen 4 support, tested on Ryzen 9 7900 > > > > Reviewed by:imp (previous version), mhorne > > Approved by:mhorne > > Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > > Differential Revision: https://reviews.freebsd.org/D41049 > > Thanks for sharing your findings. > > Now I probably know why my old kernel from stable/13 no longer booted > after updating to stable/14. I've create a new kernel config and > forgot to add "device amdtemp" & "device amdsmn" and forgot about the > issue. After removing only "device amdtemp" from my old kernel config > it boots again. > > Unfortunately reverting this commit (git revert -n 323a94afb623) > doesn't resolve this issue. Old kernel does not boot if "device > amdtemp" is enabled. Probably wrong commit or I am doing somethig > wrong!? > Strange. My FreeBSD-14 kernel boots with device amdtemp (which automatically results in amdsmn being included). It's FreeBSD-15 which fails for me. -- Gary Jennejohn
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, Sep 1, 2023, 5:03 AM Gary Jennejohn wrote: > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. > > These are respectively Zen 1 and Zen 2 CPUs. > > I built a kernel on both computers using the FreeBSD-15 source tree. > > If I include the amdtemp device in my kernel file BOTH computers end up > with a kernel panic while trying to attach the amdtemp device. > Traceback? Warner If I remove amdtemp both computers boot without any issues. > > I suspect that this commit is the cause: > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > Author: Akio Morita > Date: Tue Aug 1 22:32:12 2023 +0200 > > amdsmn(4), amdtemp(4): add support for Zen 4 > > Zen 4 support, tested on Ryzen 9 7900 > > Reviewed by:imp (previous version), mhorne > Approved by:mhorne > Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > Differential Revision: https://reviews.freebsd.org/D41049 > > -- > Gary Jennejohn > >
Re: FreeBSD-15 kernel panic when the amdtemp device is in the kernel
On Fri, 01 Sep 2023 13:03:14 +0200, Gary Jennejohn wrote: > > I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. > > These are respectively Zen 1 and Zen 2 CPUs. > > I built a kernel on both computers using the FreeBSD-15 source tree. > > If I include the amdtemp device in my kernel file BOTH computers end up > with a kernel panic while trying to attach the amdtemp device. > > If I remove amdtemp both computers boot without any issues. > > I suspect that this commit is the cause: > > commit 323a94afb6236bcec3a07721566aec6f2ea2b209 > Author: Akio Morita > Date: Tue Aug 1 22:32:12 2023 +0200 > > amdsmn(4), amdtemp(4): add support for Zen 4 > > Zen 4 support, tested on Ryzen 9 7900 > > Reviewed by:imp (previous version), mhorne > Approved by:mhorne > Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 > Differential Revision: https://reviews.freebsd.org/D41049 Thanks for sharing your findings. Now I probably know why my old kernel from stable/13 no longer booted after updating to stable/14. I've create a new kernel config and forgot to add "device amdtemp" & "device amdsmn" and forgot about the issue. After removing only "device amdtemp" from my old kernel config it boots again. Unfortunately reverting this commit (git revert -n 323a94afb623) doesn't resolve this issue. Old kernel does not boot if "device amdtemp" is enabled. Probably wrong commit or I am doing somethig wrong!? -- Herbert
FreeBSD-15 kernel panic when the amdtemp device is in the kernel
I have a laptop wioth a AMD Ryzen 5 and a tower with a AMD Ryzen 7 3700X. These are respectively Zen 1 and Zen 2 CPUs. I built a kernel on both computers using the FreeBSD-15 source tree. If I include the amdtemp device in my kernel file BOTH computers end up with a kernel panic while trying to attach the amdtemp device. If I remove amdtemp both computers boot without any issues. I suspect that this commit is the cause: commit 323a94afb6236bcec3a07721566aec6f2ea2b209 Author: Akio Morita Date: Tue Aug 1 22:32:12 2023 +0200 amdsmn(4), amdtemp(4): add support for Zen 4 Zen 4 support, tested on Ryzen 9 7900 Reviewed by:imp (previous version), mhorne Approved by:mhorne Obtained from: http://jyurai.ddo.jp/~amorita/diary/?date=20221102#p01 Differential Revision: https://reviews.freebsd.org/D41049 -- Gary Jennejohn
