Hello,
I installed a trial version of Gluster Storage Software Appliance last week.
At the moment I do some performance tests on the server.
I use Ubuntu Server 11.10 as OS.
Hardware (2x):
-AMD Opteron Quad Core 2.2GHz
-16GB RAM
-12x WD 2TB HDD @ 7200rpm
-10Gbit link between the servers
I use zfs as filesystem.
Glusterfs version 3.2.1, but I also installed 3.2.4 for testing (same behaviour).
ZFS is mounted to /data:
dd if=/dev/zero of=/data/LUN0 bs=1M count=20240
21223178240 Bytes (21 GB) kopiert, 13.6647 s, 1.6 GB/s
create a glusterfs volume (without replication for testing local speed):
gluster volume create test-volume san0:/data/test-volume
and mount:
mount -t glusterfs san0:/test-volume /mnt
dd if=/dev/zero of=/mnt/LUN0 bs=1M count=20240
21223178240 Bytes (21 GB) kopiert, 79.0924 s, 268 MB/s
I also do some tests with bonnie++. The result you can find as attachment.
As you can see, glusterfs mount is much slower than the local zfs mount.
During the dd and bonnie++ tests (glusterfs mount) the cpu load for glusterd and glusterfs processes is much more than 100% (150%-170%)
Is this the normal behaviour of glusterfs?? Or Is the Quad core cpu to slow??
Thank you for help.
Regards,
Erik Schwalbe
I installed a trial version of Gluster Storage Software Appliance last week.
At the moment I do some performance tests on the server.
I use Ubuntu Server 11.10 as OS.
Hardware (2x):
-AMD Opteron Quad Core 2.2GHz
-16GB RAM
-12x WD 2TB HDD @ 7200rpm
-10Gbit link between the servers
I use zfs as filesystem.
Glusterfs version 3.2.1, but I also installed 3.2.4 for testing (same behaviour).
ZFS is mounted to /data:
dd if=/dev/zero of=/data/LUN0 bs=1M count=20240
21223178240 Bytes (21 GB) kopiert, 13.6647 s, 1.6 GB/s
create a glusterfs volume (without replication for testing local speed):
gluster volume create test-volume san0:/data/test-volume
and mount:
mount -t glusterfs san0:/test-volume /mnt
dd if=/dev/zero of=/mnt/LUN0 bs=1M count=20240
21223178240 Bytes (21 GB) kopiert, 79.0924 s, 268 MB/s
I also do some tests with bonnie++. The result you can find as attachment.
As you can see, glusterfs mount is much slower than the local zfs mount.
During the dd and bonnie++ tests (glusterfs mount) the cpu load for glusterd and glusterfs processes is much more than 100% (150%-170%)
Is this the normal behaviour of glusterfs?? Or Is the Quad core cpu to slow??
Thank you for help.
Regards,
Erik Schwalbe
Erik Schwalbe
Canoo Engineering AG
Kirschgartenstrasse 5
CH-4051 Basel
Canoo Engineering AG
Kirschgartenstrasse 5
CH-4051 Basel
Tel: +41 61 228 94 44
Fax: +41 61 228 94 49
Fax: +41 61 228 94 49
| Version 1.96 | Sequential Output | Sequential Input | Random Seeks | Sequential Create | Random Create | |||||||||||||||||||||
| Size | Per Char | Block | Rewrite | Per Char | Block | Num Files | Create | Read | Delete | Create | Read | Delete | ||||||||||||||
| K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | |||
| san1 | 32G | 109 | 98 | 348546 | 55 | 198110 | 75 | 272 | 99 | 881967 | 78 | 1079 | 61 | 10 | 2410 | 21 | +++++ | +++ | 7343 | 52 | 4547 | 50 | +++++ | +++ | 4476 | 34 |
| Latency | 156ms | 1386ms | 1414ms | 59563us | 29339us | 39013us | Latency | 2685ms | 448us | 2168us | 44301us | 137us | 403ms | |||||||||||||
Gluster mount (local):
| Version 1.96 | Sequential Output | Sequential Input | Random Seeks | Sequential Create | Random Create | |||||||||||||||||||||
| Size | Per Char | Block | Rewrite | Per Char | Block | Num Files | Create | Read | Delete | Create | Read | Delete | ||||||||||||||
| K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | |||
| san1 | 32G | 5 | 4 | 35486 | 6 | 37397 | 9 | 3417 | 96 | 352573 | 27 | 689.9 | 11 | 10 | 563 | 2 | 8429 | 6 | 1389 | 3 | 973 | 5 | 4598 | 3 | 1736 | 4 |
| Latency | 1656ms | 301ms | 399ms | 34965us | 237ms | 81405us | Latency | 5289ms | 31277us | 25178us | 156ms | 8386us | 20471us | |||||||||||||
NFS mount (client):
| Version 1.96 | Sequential Output | Sequential Input | Random Seeks | Sequential Create | Random Create | |||||||||||||||||||||
| Size | Per Char | Block | Rewrite | Per Char | Block | Num Files | Create | Read | Delete | Create | Read | Delete | ||||||||||||||
| K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | |||
| klondike | 48G | 1114 | 98 | 105824 | 11 | 54492 | 10 | 3214 | 97 | 116006 | 9 | 775.3 | 31 | 10 | 425 | 2 | 1181 | 4 | 1012 | 4 | 519 | 3 | 2672 | 3 | 808 | 3 |
| Latency | 7768us | 29691ms | 29046ms | 19191us | 69906us | 161ms | Latency | 10507ms | 422ms | 432ms | 30837us | 3541us | 15201us | |||||||||||||
gluster mount (client):
| Version 1.96 | Sequential Output | Sequential Input | Random Seeks | Sequential Create | Random Create | |||||||||||||||||||||
| Size | Per Char | Block | Rewrite | Per Char | Block | Num Files | Create | Read | Delete | Create | Read | Delete | ||||||||||||||
| K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | K/sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | /sec | % CPU | |||
| klondike | 48G | 12 | 13 | 85126 | 18 | 38502 | 11 | 3433 | 99 | 97440 | 7 | 746.9 | 11 | 10 | 360 | 2 | 6283 | 7 | 846 | 3 | 514 | 3 | 2625 | 3 | 797 | 3 |
| Latency | 998ms | 233ms | 167ms | 8696us | 203ms | 31505us | Latency | 6475ms | 7042us | 26688us | 29773us | 22902us | 17152us | |||||||||||||
_______________________________________________ Gluster-users mailing list [email protected] http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
