Re: Where has my gbde write performance gone?

2010-05-18 Thread Joseph Gleason
For whatever it is worth, if I use geli rather than gbde I get normal
(~30MB/s) performance.

I also get the same slow gbde performance on 8.1-PRERELEASE as of last night.

I've make a kernel swaping in files from 7.2 source to see if I got
any improvement.
I pulled in:
geom_dev.c (with some hacks to get it compile)
geom_slice.c
geom_io.c

None of those improved performance.

If anyone has any suggestions for things to try, let me know.  I am
fine with switching to geli for some applications but I have about
12TB under gbde.  That would be somewhat of a bear to copy over into
geli.


On Mon, May 17, 2010 at 7:32 PM, Joseph Gleason fired...@gmail.com wrote:
 Sometime between FreeBSD 7.2-RELEASE-p4 and 8.0-RELEASE write
 performance of gbde encrypted devices seems to have dropped
 significantly.  A system I have running 7.2 seems to run gbde drives
 at or near the drive max rate (30-40MB/s) while I am seeing less than
 10% of that on 8.0 systems.

 I get the same slow writes on 8.0-RELEASE-p2 as well as 8.0-RELEASE.

 Here is an example on a fresh 8.0 install which shows gbde taking the
 drive write performance of 40 MB/s down to 2.6 MB/s:

 lab# uname -a
 FreeBSD lab.int.fireduck.com 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat
 Nov 21 15:02:08 UTC 2009
 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

 lab# dd if=/dev/urandom of=/dev/ad4s1d bs=32k count=32k
 32768+0 records in
 32768+0 records out
 1073741824 bytes transferred in 25.130537 secs (42726577 bytes/sec)

 lab# gbde init /dev/ad4s1d
 Enter new passphrase:
 Reenter new passphrase:

 lab# gbde attach /dev/ad4s1d
 Enter passphrase:

 lab# dd if=/dev/urandom of=/dev/ad4s1d.bde bs=32k count=32k
 32768+0 records in
 32768+0 records out
 1073741824 bytes transferred in 401.097004 secs (2677013 bytes/sec)

 iostat from while that last 'dd' was running:

       tty             ad4             cpu
  tin  tout  KB/t tps  MB/s  us ni sy in id
   0    22  5.67 483  2.67   0  0  4  1 96
   0    66  5.67 509  2.82   0  0  4  1 95
   0    22  5.69 514  2.86   0  0  6  1 94
   0    22  5.67 506  2.80   0  0  6  1 93
   0    22  5.67 472  2.61   0  0  4  1 95


 iostat on a FreeBSD 7.2-RELEASE-p4 box doing a similar operation:

  tin  tout  KB/t tps  MB/s  us ni sy in id
   0   22 29.54 1208 34.86   3  0 56  2 39
   0   22 29.56 1177 33.97   3  0 57  1 39
   0   22 29.54 1201 34.64   3  0 58  2 37
   0   22 29.57 1144 33.04   2  0 51  3 44
   0   22 29.56 1126 32.52   3  0 54  2 42
   0   22 29.53 1179 34.01   3  0 53  2 42
   0   22 29.57 1165 33.65   2  0 58  2 38

 One thing I notice is the larger block size the 7.2 writes but I don't
 imagine that would be that significant.

 I've been using FreeBSD in various amateurish and wrong ways since
 2.2, so I wouldn't rule out me doing something stupid.  If so, I'd
 love to know what.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org


Where has my gbde write performance gone?

2010-05-17 Thread Joseph Gleason
Sometime between FreeBSD 7.2-RELEASE-p4 and 8.0-RELEASE write
performance of gbde encrypted devices seems to have dropped
significantly.  A system I have running 7.2 seems to run gbde drives
at or near the drive max rate (30-40MB/s) while I am seeing less than
10% of that on 8.0 systems.

I get the same slow writes on 8.0-RELEASE-p2 as well as 8.0-RELEASE.

Here is an example on a fresh 8.0 install which shows gbde taking the
drive write performance of 40 MB/s down to 2.6 MB/s:

lab# uname -a
FreeBSD lab.int.fireduck.com 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat
Nov 21 15:02:08 UTC 2009
r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

lab# dd if=/dev/urandom of=/dev/ad4s1d bs=32k count=32k
32768+0 records in
32768+0 records out
1073741824 bytes transferred in 25.130537 secs (42726577 bytes/sec)

lab# gbde init /dev/ad4s1d
Enter new passphrase:
Reenter new passphrase:

lab# gbde attach /dev/ad4s1d
Enter passphrase:

lab# dd if=/dev/urandom of=/dev/ad4s1d.bde bs=32k count=32k
32768+0 records in
32768+0 records out
1073741824 bytes transferred in 401.097004 secs (2677013 bytes/sec)

iostat from while that last 'dd' was running:

   tty ad4 cpu
 tin  tout  KB/t tps  MB/s  us ni sy in id
   022  5.67 483  2.67   0  0  4  1 96
   066  5.67 509  2.82   0  0  4  1 95
   022  5.69 514  2.86   0  0  6  1 94
   022  5.67 506  2.80   0  0  6  1 93
   022  5.67 472  2.61   0  0  4  1 95


iostat on a FreeBSD 7.2-RELEASE-p4 box doing a similar operation:

 tin  tout  KB/t tps  MB/s  us ni sy in id
   0   22 29.54 1208 34.86   3  0 56  2 39
   0   22 29.56 1177 33.97   3  0 57  1 39
   0   22 29.54 1201 34.64   3  0 58  2 37
   0   22 29.57 1144 33.04   2  0 51  3 44
   0   22 29.56 1126 32.52   3  0 54  2 42
   0   22 29.53 1179 34.01   3  0 53  2 42
   0   22 29.57 1165 33.65   2  0 58  2 38

One thing I notice is the larger block size the 7.2 writes but I don't
imagine that would be that significant.

I've been using FreeBSD in various amateurish and wrong ways since
2.2, so I wouldn't rule out me doing something stupid.  If so, I'd
love to know what.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org