ZFS sends TIRMs to agressively? (Was: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system)
Hello Lev, Saturday, December 8, 2018, 7:58:37 PM, you wrote: >> Can you please narrow the problem down to a specific kernel revision? > I'm still not sure it is software or hardware problem. Looks like Samsung 850 EVO doesn't like TRIMs sent by ZFS (and I've thought it is good SSD, consumer-grade, but really good one!). I've tuned down TRIMs with vfs.zfs.per_txg_dirty_frees_percent=10 vfs.zfs.free_max_blocks=1000 vfs.zfs.vdev.trim_max_active=4 And it MOSTLY solved problem: there are some freezing from time to time (and strange consumption of CPU by low-profile threads) with these settings. When I've disabled TRIM completely all freezes are gone, and low-profile threads consume tenths of percent of CPU, as it is intended. -- Best regards, Levmailto:l...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Mateusz, Saturday, December 8, 2018, 5:27:42 PM, you wrote: >> Looks like each next compiler invocation is slower and more stressful than >> previous one. > Is this a fresh install? Almost fresh. It was installed from some rather fresh 13 snapshot and then upgraded to r341157 and custom kernel via source update. Now I'm trying to update it second time without luck. First upgrade was not so painful, as far as I can remember :-) > Can you please narrow the problem down to a specific kernel revision? I'm still not sure it is software or hardware problem. > Most importantly, does this show up with a 12.0 kernel? I didn't tried 12 kernel on this hardware. > I'm running one amd box and a number of intel boxes with various cpus, > no issues. Me too, but this is only one box which have 13 and try to compile something, all other boxes are either 11/12 or are small NanoBSD installations without toolchain... -- Best regards, Levmailto:l...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
On 12/8/18, Lev Serebryakov wrote: > Hello Lev, > > Saturday, December 8, 2018, 2:13:03 PM, you wrote: > > >> Even when build is single-job, system becomes unresponsive. With >> 4-job build running it could takes up to minute to switch screen's >> windows! > And even with 1-job kernel build upsmon's connection to remote upsd > flickers! Unbelievable. > > Looks like each next compiler invocation is slower and more stressful than > previous one. > Is this a fresh install? Can you please narrow the problem down to a specific kernel revision? Most importantly, does this show up with a 12.0 kernel? I'm running one amd box and a number of intel boxes with various cpus, no issues. -- Mateusz Guzik ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
08.12.2018 18:13, Lev Serebryakov wrote: > I'm completely lost. Is it problem of software? Hardware? If it is > hardware problem what should I blame? Try using different kern.timecounter.hardware and/or kern.eventtimer.timer but first try kern.eventtimer.periodic=1 instead of default 0. If something of this helps, try going back to defaults and then disable power-saving settings, if any. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Eugene, Saturday, December 8, 2018, 4:27:13 PM, you wrote: >> I'm completely lost. Is it problem of software? Hardware? If it is >> hardware problem what should I blame? > Try using different kern.timecounter.hardware and/or kern.eventtimer.timer > but first try kern.eventtimer.periodic=1 instead of default 0. Nothing helps. I've tried periodic=1 and replace hardware and time with HPT (from TSC-Low and LAPIC), but system still "sticky" with single-job build and unresposnive with multiple-job build, and still there is strange bursts of CPU consumption from threads and processes which should be low-profile. > If something of this helps, try going back to defaults and then disable > power-saving settings, if any. I'll try to disable C2/C3 and turn off Turbo as next step... -- Best regards, Levmailto:l...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Lev, Saturday, December 8, 2018, 2:13:03 PM, you wrote: > Even when build is single-job, system becomes unresponsive. With > 4-job build running it could takes up to minute to switch screen's windows! And even with 1-job kernel build upsmon's connection to remote upsd flickers! Unbelievable. Looks like each next compiler invocation is slower and more stressful than previous one. -- Best regards, Levmailto:l...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Lev,
Saturday, December 8, 2018, 2:13:03 PM, you wrote:
> Another strange thing I noticed: when system is in such state, "top -SH"
> shows that sometimes very low-profile processes, like clock software
> interrupt (!) could consume large amount of CPU for short periods time. When
> system is idle there never will be "intr{swi4: clock (0)}" consuming 55% CPU
> for one "frame" or sshd, or screen itself.
Like this. This system doesn't have any significant network traffic now —
only one ssh connection, which is used as console. And 62.3% for network
card. WTF?!
PID USERNAMEPRI NICE SIZERES STATEC TIMEWCPU COMMAND
20128 root1010 104M74M CPU1 1 0:31 100.00% cc
0 root-76- 0 4608K -2 53:25 62.23%
kernel{if_config_tqg_0}
11 root-60- 0 240K WAIT 0 25:45 24.89% intr{swi4:
clock (0)}
9 root -8- 0 160K tx->tx 0 7:38 24.88%
zfskern{txg_thread_enter}
995 root 24017M 7676K select 1 2:20 12.44% sendmail
13791 root 24024M15M select 0 0:04 12.44% make
--
Best regards,
Levmailto:l...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
‐‐‐ Original Message ‐‐‐
On Saturday, December 8, 2018 8:18 AM, Lev Serebryakov wrote:
> Hello Lev,
>
> Saturday, December 8, 2018, 2:13:03 PM, you wrote:
>
> > Another strange thing I noticed: when system is in such state, "top -SH"
> > shows that sometimes very low-profile processes, like clock software
> > interrupt (!) could consume large amount of CPU for short periods time. When
> > system is idle there never will be "intr{swi4: clock (0)}" consuming 55% CPU
> > for one "frame" or sshd, or screen itself.
>
> Like this. This system doesn't have any significant network traffic now —
> only one ssh connection, which is used as console. And 62.3% for network
> card. WTF?!
>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 20128 root 101 0 104M 74M CPU1 1 0:31 100.00% cc
> 0 root -76 - 0 4608K - 2 53:25 62.23% kernel{if_config_tqg_0}
> 11 root -60 - 0 240K WAIT 0 25:45 24.89% intr{swi4: clock (0)}
> 9 root -8 - 0 160K tx->tx 0 7:38 24.88% zfskern{txg_thread_enter}
>
> 995 root 24 0 17M 7676K select 1 2:20 12.44% sendmail
> 13791 root 24 0 24M 15M select 0 0:04 12.44% make
>
>
> -
>
> Best regards,
> Lev mailto:l...@freebsd.org
>
> freebsd-hack...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
I had super slow build for r341270, but I thought it was because I accidentally
left WITNESS option set.
I killed it after about 10 hours, booted to single user and rebuilt kernel
there.
Waitman
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Lev, Saturday, December 8, 2018, 2:13:03 PM, you wrote: > I've checked all "standard" places — CPU is not throttling, SSD looks > perfectly Ok according to SMART and there is no complains from AHCI driver > about timeouts and such, system doesn't start to use swap. ZFS ARC was checked too. Here is statistics from top when single-job kernel build is in action. A lot of free memory, small ARC, too much CPU is consumed by interrupts, but there is free CPU clocks: last pid: 19488; load averages: 7.03, 5.35, 5.10 up 0+14:43:04 15:09:55 417 threads: 7 running, 395 sleeping, 15 waiting CPU 0: 50.0% user, 0.0% nice, 0.0% system, 16.4% interrupt, 33.6% idle CPU 1: 16.4% user, 0.0% nice, 16.8% system, 0.0% interrupt, 66.8% idle CPU 2: 0.0% user, 0.0% nice, 33.2% system, 0.0% interrupt, 66.8% idle CPU 3: 33.2% user, 0.0% nice, 33.2% system, 0.0% interrupt, 33.6% idle Mem: 28M Active, 315M Inact, 2076K Laundry, 2541M Wired, 1129K Buf, 5031M Free ARC: 1025M Total, 197M MFU, 415M MRU, 514K Anon, 20M Header, 392M Other 189M Compressed, 563M Uncompressed, 2.98:1 Ratio Swap: 16G Total, 16G Free -- Best regards, Levmailto:l...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Painfully slow compilation (read: "make buildworld buildkernel") on not-so-weak system
Hello Freebsd-hackers,
I'm experiencing very strange situation on my lab system which is
E3-1220v2, 8GiB of RAM and 850 EVO SATA SSD (with single ZFS pool).
It runs CURRENT r341157. Kernel is built *without* INVARIANTS and other
heavy debug aids.
Everything works great — but compilation. "make -j *1* buildkernel" takes
forever and each compiler invocation takes up to 10 seconds. For example,
I've clocked compilation of sys/dev/aic7xxx/aic7xxx_93cx6.c by stopwatch and
it takes 9 seconds. Please note, it is SINGLE JOB build. If I run "make
-j4" it will be much longer for each compiler out of 4. And all this time
"cc" / "c++" consume 100% of CPU.
Even when build is single-job, system becomes unresponsive. With
4-job build running it could takes up to minute to switch screen's windows!
Another strange thing I noticed: when system is in such state, "top -SH"
shows that sometimes very low-profile processes, like clock software
interrupt (!) could consume large amount of CPU for short periods time. When
system is idle there never will be "intr{swi4: clock (0)}" consuming 55% CPU
for one "frame" or sshd, or screen itself.
I'm completely lost. Is it problem of software? Hardware? If it is
hardware problem what should I blame?
I've checked all "standard" places — CPU is not throttling, SSD looks
perfectly Ok according to SMART and there is no complains from AHCI driver
about timeouts and such, system doesn't start to use swap.
--
Best regards,
Lev mailto:l...@freebsd.org
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
