[Gluster-users] Performance problems with glusterfs

[Erik Schwalbe]

Hello, 

I installed a trial version of Gluster Storage Software Appliance last week. 
At the moment I do some performance tests on the server. 
I use Ubuntu Server 11.10 as OS. 
Hardware (2x): 
-AMD Opteron Quad Core 2.2GHz 
-16GB RAM 
-12x WD 2TB HDD @ 7200rpm 
-10Gbit link between the servers 

I use zfs as filesystem. 
Glusterfs version 3.2.1, but I also installed 3.2.4 for testing (same 
behaviour). 

ZFS is mounted to /data: 
dd if=/dev/zero of=/data/LUN0 bs=1M count=20240 
21223178240 Bytes (21 GB) kopiert, 13.6647 s, 1.6 GB/s 

create a glusterfs volume (without replication for testing local speed): 
gluster volume create test-volume san0:/data/test-volume 

and mount: 
mount -t glusterfs san0:/test-volume /mnt 

dd if=/dev/zero of=/mnt/LUN0 bs=1M count=20240 
21223178240 Bytes (21 GB) kopiert, 79.0924 s, 268 MB/s 

I also do some tests with bonnie++. The result you can find as attachment. 

As you can see, glusterfs mount is much slower than the local zfs mount. 
During the dd and bonnie++ tests (glusterfs mount) the cpu load for glusterd 
and glusterfs processes is much more than 100% (150%-170%) 

Is this the normal behaviour of glusterfs?? Or Is the Quad core cpu to slow?? 

Thank you for help. 

Regards, 
Erik Schwalbe 

Title: Bonnie++ Benchmark results

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Size

Per Char

Block

Rewrite

Per Char

Block

Num Files

Create

Read

Delete

Create

Read

Delete

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

san1

32G

109

98

348546

55

198110

75

272

99

881967

78

1079

61

10

2410

21

+++++

+++

7343

52

4547

50

+++++

+++

4476

34

Latency

156ms

1386ms

1414ms

59563us

29339us

39013us

Latency

2685ms

448us

2168us

44301us

137us

403ms

Gluster mount (local):

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Size

Per Char

Block

Rewrite

Per Char

Block

Num Files

Create

Read

Delete

Create

Read

Delete

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

san1

32G

5

4

35486

6

37397

9

3417

96

352573

27

689.9

11

10

563

2

8429

6

1389

3

973

5

4598

3

1736

4

Latency

1656ms

301ms

399ms

34965us

237ms

81405us

Latency

5289ms

31277us

25178us

156ms

8386us

20471us

NFS mount (client):

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Size

Per Char

Block

Rewrite

Per Char

Block

Num Files

Create

Read

Delete

Create

Read

Delete

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

klondike

48G

1114

98

105824

11

54492

10

3214

97

116006

9

775.3

31

10

425

2

1181

4

1012

4

519

3

2672

3

808

3

Latency

7768us

29691ms

29046ms

19191us

69906us

161ms

Latency

10507ms

422ms

432ms

30837us

3541us

15201us

gluster mount (client):

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Size

Per Char

Block

Rewrite

Per Char

Block

Num Files

Create

Read

Delete

Create

Read

Delete

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

klondike

48G

12

13

85126

18

38502

11

3433

99

97440

7

746.9

11

10

360

2

6283

7

846

3

514

3

2625

3

797

3

Latency

998ms

233ms

167ms

8696us

203ms

31505us

Latency

6475ms

7042us

26688us

29773us

22902us

17152us

_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users