Andrea, Interesting configuration. Is there a particular reason why you decided on using Xen VMs? Is failover better with Xen instances? I'm guessing you don't have hundreds of clients hammering the hardware.
----- "Andrea Rucks" <[email protected]> wrote: > In your thread discussing "e2scan MDT backup", I was very interested > in finding out more about how you initially setup your OST's and MDT's > with LVM. We're implementing a new set of production Lustre servers > and had Sun come on-site last fall to help get us started. Our Sun > Lustre consultant was very knowledgeable and helpful and he setup our > environment as follows: <SNIP!> > As you can see, our Sun consultant set our production environment up > to use standard disk partitions that are striped across two sets of > servers for our five Lustre filesystems (which is working fine and we > can grow it no problems). We're running RHEL 5.2 XEN MGS/MDT/OSS > servers using Linux Heartbeat for STONITH and failover. The XEN domU's > are LVM-backed on EMC SAN. Lustre was compiled into the XEN kernel and > we've compiled and are using the patchless Lustre client (Lustre > 1.6.6). I'm curious as to why you created 5 filesystems on the same "hardware" instead of one big filesystem? > What I'm really curious about is how to go about setting up this > environment under LVM2. Is it as simple as taking the "xvd?" disk on > the OSS and MDS servers, doing a pvcreate, vgcreate, lvcreate, mkfs? > Do you have to be careful about anything (striping as seen above)? May > I ask if you might be able to give an example? I'd like to test LVM in > our Lustre certification environment, but would like to make sure I > have an idea of what I'm doing before I mess up my working cert > environment. Are there any pros / cons for LVM2 versus disk > partitions? Backup issues, speed, performance, etc? The write performance does suffer as you add snapshots but on the MDT it is unlikely to be that noticeable and you can remove the snapshot when you have grabbed what you need for the EA and file backups. Also you shouldn't need to put the OSTs on LVM if you just want to do MDT backups. OST snapshots are only useful if you want to mount a snapshot of the entire lustre filesystem on a client. Beware though that this is likely to slow down the production filesystem permanently. We run this configuration on a backup filesystem that is not accessed by production so the slowdown is not important. I'm not sure it is possible to migrate an existing filesystem to LVM easily - you would need to do a file backup of your MDT first and restore to the LVM device (section 15.1.3.1 of the manual). So in your case to wipe (!) a single MDT and create a new one I'd do something like: pvcreate /dev/xvdj vgcreate lusfs01 /dev/xvdj vgchange -a y lusfs01 lvcreate -L3G -nmdt lusfs01 Now you do the mkfs.lustre stuff on it and the corresponding OST devices. You can then create and remove a snapshot at anytime with something like: lvcreate -Ay -nmdt-snap -L50G -s -prw /dev/lusfs01/mdt lvremove /dev/lusfs01/mdt-snap I hope this helps. Daire _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
