Re: [Lustre-discuss] Failover for MGS

Mon, 12 Nov 2007 14:23:40 -0800 

Hi,
Thanks for that. Actually I have a little more complex situation here. I have two sets of clients. First set is working in 10.142.10.0/24 network and the second set is working in 10.143.0.0/16 network. 
Each server has two NIC's.
NIC1 = ETH0 10.143.0.0/16 and NIC2= ETH1 10.142.10.0/24
lnet configures network in the following manner:
eth0 = <ip>@tcp0
eth1 = <ip>@tcp1
I am going to change lustre configuration in order to introduce failover features. 
MGS is cobined with with mdt01=/dev/dm-0

on mds01
tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-0 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-1 

on oss1
tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-0 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-1 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-2 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-3 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-4 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-5 

on oss2
tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-6 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-7 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-8 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-9 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-10 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-11 

on oss3
tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-0 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-1 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-2 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-3 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-4 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-5 

on oss4
tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-6 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-7 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-8 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-9 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-10 tunefs.lustre --erase-params --writeconf -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] -- [EMAIL PROTECTED],[EMAIL PROTECTED] /dev/dm-11 

Will above be correct?

Cheers,

Wojciech Turek


On 12 Nov 2007, at 21:36, Robert LeBlanc wrote:


You should just unmount all the clients, all OSTs and then:

tunefs.lustre —failnode [EMAIL PROTECTED] —writeconf /dev/shared/disk
If your volume is already on the shared disk, them mount everything and you should be good to go. You can also do it on a live mounted system by using lctl, but I’m not exactly sure how to do that. 

Robert

On 11/12/07 2:24 PM, "Wojciech Turek" <[EMAIL PROTECTED]> wrote:


Hi,
How will look my tunefs.lustre command line if I would like to configure failnode for my MDS. I have two MDT's and MGS is on the same block device that one of MDT's ? I have also two servers connected to share matadata storage. 

Thanks,

Wojciech
On 12 Nov 2007, at 20:49, Nathan Rutman wrote:


Robert LeBlanc wrote:
Ok, I feel really stupid. I've done this before without any problem, but I can't seem to get it to work and I can't find my notes from the last time I did it. We have separate MGS and MDTs. I can't seem to get our MGS to 
failover correctly after reformatting it.

mkfs.lustre --mkfsoptions="-O dir_index" --reformat --mgs
[EMAIL PROTECTED] /dev/mapper/ldiskc-part1
The MGS doesn't actually use the --failnode option (although it won't 
hurt).  You actually have to tell the other nodes
in the system (servers and clients) about the failover options for the 
MGS (use the --mgsnode parameter on servers, and mount address for
clients). The reason is because the servers must contact the MGS for 
the configuration information, and they can't ask the MGS where its
failover partner is if e.g. the failover partner is the one that's running.
We are running this on Debian, using the Lustre 1.6.3 debs from svn on Lenny with 2.6.22.12. I've tried several permutations of the mkfs.lustre command, specifing both nodes as failover, and both nodes as MGS and pretty much every other combination of the above. With the above command tunefs.lustre 
shows that failnode and mgsnode are the failover node.

Thanks,
Robert

Robert LeBlanc
College of Life Sciences Computer Support
Brigham Young University
[EMAIL PROTECTED]
(801)422-1882


_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss



_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss



Mr Wojciech Turek
Assistant System Manager
University of Cambridge
High Performance Computing service
email: [EMAIL PROTECTED]
tel. +441223763517








Robert LeBlanc
College of Life Sciences Computer Support
Brigham Young University
[EMAIL PROTECTED]
(801)422-1882



Mr Wojciech Turek
Assistant System Manager
University of Cambridge
High Performance Computing service
email: [EMAIL PROTECTED]
tel. +441223763517




_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss

Reply via email to