Re: dmesg hangs 7.4
On Sunday, 10 March 2024 at 11:09, Tobias Fiebig wrote: > > Would still give it a try, esp. given that a large text file cat also > shows this MTU-y behavior. ;-) > > Still, I acknowledge that I do have a very MTU-hammer-view of network > things a lot. > > In any case, the issue sounds interesting, and I am curious what you > will ultimately find. > > I'm still working on various avenues of possibility, the network switch the BSD box is attached to is one of them, but unlikely becuase all other traffic transiting through the switch seems to be just fine.
Re: dmesg hangs 7.4
Thanks for the suggestions Stuart, I'll work on more experimentation next week. On Saturday, 9 March 2024 at 18:00, Stuart Henderson wrote: > Can you get dmesg out by another means (most likely, write to a usb stick)? > > Any difference if you install a snapshot? > > Anything special with the network setup? > > Anything odd in dmesg on the box you're ssh'ing from? > > On 2024-03-09, Laura Smith n5d9xq3ti233xiyif...@protonmail.ch wrote: > > > Hi > > > > I've got a fresh install of 7.4 on a new box and am seeing a very weird > > problem. > > > > If I enter "dmesg" I get a few lines of output and then it hangs and my ssh > > connection gets dropped. I ran syspatch, rebooted and the problem persists. > > > > Example: > > > > # dmesg > > MX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU,WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1,XSAVES > > cpu19: 32KB 64b/line 8-way D-cache, 64KB 64b/line 8-way I-cache, 2MB > > 64b/line 16-way L2 cache, 30MB 64b/line 12-way L3 cache > > cpu19: smt 0, core 35, package 0 > > cpu20 at mainbus0: apid 72 (application processor) > > cpu20: 12th Gen Intel(R) Core(TM) i9-12900TE, 3392.18 MHz, 06-97-02, patch > > 0025 > > Timeout, server 10.1.2.3 not responding. > > > > -- > Please keep replies on the mailing list.
Re: dmesg hangs 7.4
Moin, On Sat, 2024-03-09 at 17:24 +, Laura Smith wrote: > Nice idea Tobias, but I forgot to mention both machines are on the > same LAN, and the LAN is operating with standard MTU, no jumbos. Would still give it a try, esp. given that a large text file cat also shows this MTU-y behavior. ;-) Still, I acknowledge that I do have a very MTU-hammer-view of network things a lot. In any case, the issue sounds interesting, and I am curious what you will ultimately find. With best regards, Tobias
Re: dmesg hangs 7.4
On 3/9/24 17:07, Laura Smith wrote: Hi I've got a fresh install of 7.4 on a new box and am seeing a very weird problem. If I enter "dmesg" I get a few lines of output and then it hangs and my ssh connection gets dropped. I ran syspatch, rebooted and the problem persists. Example: # dmesg MX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU,WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu19: 32KB 64b/line 8-way D-cache, 64KB 64b/line 8-way I-cache, 2MB 64b/line 16-way L2 cache, 30MB 64b/line 12-way L3 cache cpu19: smt 0, core 35, package 0 cpu20 at mainbus0: apid 72 (application processor) cpu20: 12th Gen Intel(R) Core(TM) i9-12900TE, 3392.18 MHz, 06-97-02, patch 0025 Timeout, server 10.1.2.3 not responding. Hi, I don't know if it's relevant but I have a VPS at openbsd.amsterdam that seems to have similar behaviour. I can type top, or any command like ps and it will hang a bit before continuing. There is a mode in SSH to enable keepalives, perhaps play with that? As per my vps, I have mentioned it to the admin of the vps (Mischa) and he made me some vps's to compare, however I'm leaving that service next month so there is really no need. I had suspected perhaps a drive failure on the RAID but Mischa had replaced the drive on that server (I think it has 12 cores or something) and it shouldn't be that, but I dunno (shrug). Do you perhaps use vmm on your 2x cpu machine? Best Regards, -pjp -- Over thirty years experience on UNIX-like Operating Systems starting with QNX.
Re: dmesg hangs 7.4
Can you get dmesg out by another means (most likely, write to a usb stick)? Any difference if you install a snapshot? Anything special with the network setup? Anything odd in dmesg on the box you're ssh'ing *from*? On 2024-03-09, Laura Smith wrote: > Hi > > I've got a fresh install of 7.4 on a new box and am seeing a very weird > problem. > > If I enter "dmesg" I get a few lines of output and then it hangs and my ssh > connection gets dropped. I ran syspatch, rebooted and the problem persists. > > Example: > > # dmesg > MX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU,WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1,XSAVES > cpu19: 32KB 64b/line 8-way D-cache, 64KB 64b/line 8-way I-cache, 2MB 64b/line > 16-way L2 cache, 30MB 64b/line 12-way L3 cache > cpu19: smt 0, core 35, package 0 > cpu20 at mainbus0: apid 72 (application processor) > cpu20: 12th Gen Intel(R) Core(TM) i9-12900TE, 3392.18 MHz, 06-97-02, patch > 0025 > Timeout, server 10.1.2.3 not responding. > > -- Please keep replies on the mailing list.
Re: dmesg hangs 7.4
Nice idea Tobias, but I forgot to mention both machines are on the same LAN, and the LAN is operating with standard MTU, no jumbos. Sent with Proton Mail secure email. On Saturday, 9 March 2024 at 17:02, Tobias Fiebig wrote: > Might be MTU? Can you try what happens with `find /` ? > > If it freezes then, too: > > On both machines: > > ping -s 1252 -D $remote_IP > > if it works, increase by half the way to 1472, try if it still works: > > ping -s 1362 -D $remote_IP > > etc. > > If it does not work, you half the distance to the previously working > one, e.g.: > > ping -s 1362 -D $remote_IP << works > > ping -s 1417 -D $remote_IP << does not work > > then next: > > ping -s 1390 -D $remote_IP > > If you use IPv6, you need to decrease the starting/end value by 20 and > use ping6, i.e., as min 1232 and as max 1452; If the hosts cannot reach > each other directly (NAT), it depends on the path which addr to use; > Would need traceroutes between the hosts then. > > The assumption would be that you do have an MTU issue on-path there, > which is not correctly signaled. This only hits as soon as the packets > get a size larger than the MTU, e.g., when typing dmesg (or find /). > > With best regards, > Tobias > > On Sat, 2024-03-09 at 16:07 +, Laura Smith wrote: > > > Hi > > > > I've got a fresh install of 7.4 on a new box and am seeing a very > > weird problem. > > > > If I enter "dmesg" I get a few lines of output and then it hangs and > > my ssh connection gets dropped. I ran syspatch, rebooted and the > > problem persists. > > > > Example: > > > > # dmesg > > MX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS- > > CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2 > > ,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,R > > DTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SM > > EP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU, > > WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_A > > LL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM > > ,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1 > > ,XSAVES > > cpu19: 32KB 64b/line 8-way D-cache, 64KB 64b/line 8-way I-cache, 2MB > > 64b/line 16-way L2 cache, 30MB 64b/line 12-way L3 cache > > cpu19: smt 0, core 35, package 0 > > cpu20 at mainbus0: apid 72 (application processor) > > cpu20: 12th Gen Intel(R) Core(TM) i9-12900TE, 3392.18 MHz, 06-97-02, > > patch 0025 > > Timeout, server 10.1.2.3 not responding.
Re: dmesg hangs 7.4
Same thing, it hangs : # cat /dev/urandom | openssl enc -base64 -out foo ^C # ls -lah foo 1.5G Mar 9 17:17 foo # cat foo Bunch of text, then hang, then dropped "Timeout, server not responding." Sent with Proton Mail secure email. On Saturday, 9 March 2024 at 16:55, Mihai Popescu wrote: > Just curious, what happens when you do: > # cat
Re: dmesg hangs 7.4
Might be MTU? Can you try what happens with `find /` ? If it freezes then, too: On both machines: ping -s 1252 -D $remote_IP if it works, increase by half the way to 1472, try if it still works: ping -s 1362 -D $remote_IP etc. If it does not work, you half the distance to the previously working one, e.g.: ping -s 1362 -D $remote_IP << works ping -s 1417 -D $remote_IP << does not work then next: ping -s 1390 -D $remote_IP If you use IPv6, you need to decrease the starting/end value by 20 and use ping6, i.e., as min 1232 and as max 1452; If the hosts cannot reach each other directly (NAT), it depends on the path which addr to use; Would need traceroutes between the hosts then. The assumption would be that you do have an MTU issue on-path there, which is not correctly signaled. This only hits as soon as the packets get a size larger than the MTU, e.g., when typing dmesg (or find /). With best regards, Tobias On Sat, 2024-03-09 at 16:07 +, Laura Smith wrote: > Hi > > I've got a fresh install of 7.4 on a new box and am seeing a very > weird problem. > > If I enter "dmesg" I get a few lines of output and then it hangs and > my ssh connection gets dropped. I ran syspatch, rebooted and the > problem persists. > > Example: > > # dmesg > MX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS- > CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2 > ,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,R > DTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SM > EP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU, > WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_A > LL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM > ,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1 > ,XSAVES > cpu19: 32KB 64b/line 8-way D-cache, 64KB 64b/line 8-way I-cache, 2MB > 64b/line 16-way L2 cache, 30MB 64b/line 12-way L3 cache > cpu19: smt 0, core 35, package 0 > cpu20 at mainbus0: apid 72 (application processor) > cpu20: 12th Gen Intel(R) Core(TM) i9-12900TE, 3392.18 MHz, 06-97-02, > patch 0025 > Timeout, server 10.1.2.3 not responding.
Re: dmesg hangs 7.4
Just curious, what happens when you do: # cat
