On Oct 18, 2023, at 13:04, Peter Grandi via lustre-discuss <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> wrote:
So I have been upgrading my one and only MDT to a larger ZFS pool, by the classic route of creating a new pool, new MDT, and then 'zfs send'/zfs receive' for the copy over (BTW for those who may not be aware 'zfs send' output can be put into a file to do offline backups of a Lustre ZFS filesystem instance). At first I just created an empty MGT on the new devices (on a server with the same NID as the old MGS), with the assumption that given that MDTs and OSTs have unique (filesystem instance, target type, index number) triples, with NIDs being just the address to find the MGS, or where they can be found, they would just register themselves with the MGT on startup. But I found that there was a complaint that they were in a registered state, and the MGT did not have their registration entries. I am not sure that is the purpose of that check. So I just copied over the old MGT where they were registered, and all was fine. * Is there a way to re-register MDTs and OSTs belonging to a given filesystem instance into a new different MGT? If you run the "writeconf" process documented in the Lustre Manual, the MDT(s) and OST(s) will re-register themselves with the MGT. * Is there a purpose to check whether MDTs and OSTs are or not registered in a given MGT? Yes, this prevents the MDTs/OSTs from accidentally becoming part of a different filesystem that might have been incorrectly formatted with the same fsname (e.g. "lustre" has been used as the fsname more than once). * Is there a downside to register MDTs and OSTs in a different MGT from that which they were registered with initially? Not too much. The new MGT will not have any of the old configuration parameters, but running "writeconf" will also reset any "conf_param" parameters so not much different (but it will not reset "set_param -P" parameters). My guess is that the MGT does not just contain the identities and addresses of MDTs and OSTs of one or more filesystem instance, but also a parameter list If so, is there are way to dump the parameter for a filesystem instance so it can be restored to a different MGT? Yes, the "lctl --device MGS llog_print CONFIG_LOG" command will dump all of the config commands for a particular MDT/OST or the "params" log for "set_param -P". The parameters can be restored from a file with "lctl set_param -F". See the lctl-set_param.8 and lctl-llog_print.8 man pages for details. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org