Re: Filesystem operations slower in 13.0 than 12.2

2021-03-23 Thread Mark Millard via freebsd-stable
On 2021-Mar-22, at 22:51, Kevin Oberman wrote: > On Mon, Mar 22, 2021 at 8:19 AM Adrian Chadd wrote: >> On Mon, 15 Mar 2021 at 14:58, Kevin Oberman wrote: >> >> > > >> > > It appears that the messages are associated with reading >> > > the disk(s), not directly with writing them, where the

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-22 Thread Kevin Oberman
On Mon, Mar 22, 2021 at 8:19 AM Adrian Chadd wrote: > On Mon, 15 Mar 2021 at 14:58, Kevin Oberman wrote: > > > > > > > It appears that the messages are associated with reading > > > the disk(s), not directly with writing them, where the > > > reads take more than "hz * 20" time units to

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-22 Thread Adrian Chadd
On Mon, 15 Mar 2021 at 14:58, Kevin Oberman wrote: > > > > It appears that the messages are associated with reading > > the disk(s), not directly with writing them, where the > > reads take more than "hz * 20" time units to complete. > > (I'm looking at main (14) code.) What might contribute > >

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-15 Thread Mark Millard via freebsd-stable
On 2021-Mar-15, at 14:57, Kevin Oberman wrote: > Responses in-line. > > On Sun, Mar 14, 2021 at 3:09 PM Mark Millard wrote: > >> On 2021-Mar-14, at 11:09, Kevin Oberman wrote: >> >> > . . . >> > >> > Seems to only occur on large r/w operations from/to the same disk. "sp >> > big-file

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-15 Thread Kevin Oberman
Responses in-line. On Sun, Mar 14, 2021 at 3:09 PM Mark Millard wrote: > > > On 2021-Mar-14, at 11:09, Kevin Oberman wrote: > > > . . . > > > > Seems to only occur on large r/w operations from/to the same disk. "sp > > big-file /other/file/on/same/disk" or tar/untar operations on large >

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-14 Thread Mark Millard via freebsd-stable
On 2021-Mar-14, at 11:09, Kevin Oberman wrote: > . . . > > Seems to only occur on large r/w operations from/to the same disk. "sp > big-file /other/file/on/same/disk" or tar/untar operations on large files. > Hit this today updating firefox. > > I/O starts at >40MB/s. Dropped to about

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-14 Thread Kevin Oberman
On Sat, Mar 13, 2021 at 9:43 PM Kevin Oberman wrote: > No improvement with stable/13-n244880-cec3990d347. It may be worse. > worse. An attempt to unpack firefox-86.0.1,2 saw disk rates in the range > of 800K to 2 MB/s range and with repeated 30 second freezes. I have no idea > what made it so

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-13 Thread Kevin Oberman
No improvement with stable/13-n244880-cec3990d347. It may be worse. worse. An attempt to unpack firefox-86.0.1,2 saw disk rates in the range of 800K to 2 MB/s range and with repeated 30 second freezes. I have no idea what made it so much worse, but I'm forced to start wondering if it could be a

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-13 Thread Warner Losh
On Sat, Mar 13, 2021 at 6:37 PM Kevin Oberman wrote: > I have been dealing with this for a long time since head back in September > through 13-stable of Mar-4. I have seen no improvement over this time. It > seems (my perception without supporting data) that it got worse in the > timeframe of

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-13 Thread Kevin Oberman
I have been dealing with this for a long time since head back in September through 13-stable of Mar-4. I have seen no improvement over this time. It seems (my perception without supporting data) that it got worse in the timeframe of BETA-3 tag. I was running stable, so not quite BETA-3. It also

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-13 Thread Warner Losh
On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman wrote: > Just spent a little time looking at my issue and have a few more notes: > What version did you evaluate? There's a number of changes lately that could have a big impact on this... Warner > Seems to only occur on large r/w operations

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-13 Thread Kevin Oberman
Just spent a little time looking at my issue and have a few more notes: Seems to only occur on large r/w operations from/to the same disk. "sp big-file /other/file/on/same/disk" or tar/untar operations on large files. Hit this today updating firefox. I/O starts at >40MB/s. Dropped to about

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Mark Millard via freebsd-stable
Konstantin Belousov kostikbel at gmail.com wrote on Fri Mar 5 23:12:13 UTC 2021 : > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: . . . > > Command: /usr/bin/time -l portsnap extract (these tests done with 2 > > different idle servers but with same 4TB HDDs models) > > >

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Christos Chatzaras
Hello Konstantin, > On 6 Mar 2021, at 01:12, Konstantin Belousov wrote: > > There was (is) bugs in FreeBSD UFS SU < 13 > - some LoR existed in SU code, where it needed to lock a containing directory > to provide posix guarantees for fsync(), while owning the vnode lock. I > do not believe it

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Konstantin Belousov
On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: > I did some more tests. Finally I see similar results (with the exception of > one "portsnap extract" test). Also with 13.0 I can't trigger a bug that I > describe here: > >

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Christos Chatzaras
I did some more tests. Finally I see similar results (with the exception of one "portsnap extract" test). Also with 13.0 I can't trigger a bug that I describe here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250576

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Mark Millard via freebsd-stable
On 2021-Mar-4, at 14:16, Mark Millard wrote: > Christos Chatzaras chris at cretaforce.gr wrote on > Thu Mar 4 21:41:01 UTC 2021 : > > >> After finding slow filesystem operations with 13.0-BETA2 I did more tests. >> >> All tests done with same hardware (Seagate ST4000NM0245 4TB HDD - 2

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-05 Thread Dewayne Geraghty
Thanks Helge, there is a 10% increase in involuntary context switching which suggests that other things were occurring on the testing platform during the 13.0B4 run or 13 has adversely changed? (And I'd discount the latter due to the voluntary CS being similar across runs) I appreciate you

Re: Filesystem operations slower in 13.0 than 12.2

2021-03-04 Thread Mark Millard via freebsd-stable
Christos Chatzaras chris at cretaforce.gr wrote on Thu Mar 4 21:41:01 UTC 2021 : > After finding slow filesystem operations with 13.0-BETA2 I did more tests. > > All tests done with same hardware (Seagate ST4000NM0245 4TB HDD - 2 disks > with RAID-1 using gmirror). > > Filesystem mounted with

Filesystem operations slower in 13.0 than 12.2

2021-03-04 Thread Christos Chatzaras
Hello, After finding slow filesystem operations with 13.0-BETA2 I did more tests. All tests done with same hardware (Seagate ST4000NM0245 4TB HDD - 2 disks with RAID-1 using gmirror). Filesystem mounted with noatime. Command used: /usr/bin/time -l portsnap extract but similar differences I