Hi,
We are testing 1.6 beta7 over IB. On a test setup (1 MDS, 1 OSS with 6
OSTs (RAID 5 each 5 drives)) we observer uneven load among the OSTs.
Testing from one to 5 clients Lustre schedules evenly (one OST per
client). With client count > 5 sometimes one OST is not used at all
(e.g. 6, 9 clients) or the utilisation is not as expected. The FS is
otherwise empty. We used IOR for testing.
Following are some examples to explain the issue. 2% are overhead used
for the empty FS.
OST usage for 9 clients (expected 3 OSTs with 2 clients, 3 OSTs with 1
client each):
16% /mnt/test/ost0 - 2 clients
2% /mnt/test/ost1 - 0 clients ***
9% /mnt/test/ost2 - 1 clients
16% /mnt/test/ost3 - 2 clients
16% /mnt/test/ost4 - 2 clients
16% /mnt/test/ost5 - 2 clients
OST usage for 12 clients (expected 6 OSTs with 2 clients each):
9% /mnt/test/ost0 - 1 clients
9% /mnt/test/ost1 - 1 clients
23% /mnt/test/ost2 - 3 clients
23% /mnt/test/ost3 - 3 clients
16% /mnt/test/ost4 - 2 clients
16% /mnt/test/ost5 - 2 clients
OST usage for 15 clients (expected 4 OSTs with 3 clients, 2 OSTs with 2
clients each):
16% /mnt/test/ost0 - 2 clients
23% /mnt/test/ost1 - 3 clients
23% /mnt/test/ost2 - 3 clients
23% /mnt/test/ost3 - 3 clients
23% /mnt/test/ost4 - 3 clients
9% /mnt/test/ost5 - 1 clients ***
OST usage for 18 clients (expected 6 OSTs with 3 clients each):
23% /mnt/test/ost0 - 3 clients
29% /mnt/test/ost1 - 4 clients
29% /mnt/test/ost2 - 4 clients
23% /mnt/test/ost3 - 3 clients
16% /mnt/test/ost4 - 2 clients
16% /mnt/test/ost5 - 2 clients
The behavior can be reproduced. Uneven OST utilisation leads to lower
than possible performance. How can we achieve better distribution over
the OSTs without manual assignment?
Is there a setting to have a round-robin scheduling for the OST to use?
---
Stripe setting:
We want to have very high performance to a single client by striping
over 6 OSTs.
What parameters should be adjusted to achieve optimal performance?
Thanks,
Mirko
_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
