I have to re-create because I borked up the underlying disks when they
were first configured, leaving a singleton disk in the pool with no
replica, and I'm also replacing them all with larger disks.
I was un-aware of the send/receive feature. I'll have to look into it.
Thanks for your input.
bob
On 1/14/2016 11:16 AM, Stearman, Marc wrote:
You should also be able to use the ZFS send/receive feature to send that pool
or data set to another node, and then send it back. It’s unclear why you are
draining it and recreating it, but if you are replacing the storage underneath,
you can send the dataset somewhere else (another OSS node perhaps), and then
send it back once you are ready.
-Marc
----
D. Marc Stearman
Lustre Operations Lead
stearm...@llnl.gov
Office: 925-423-9670
Mobile: 925-216-7516
On Jan 13, 2016, at 7:14 PM, Dilger, Andreas <andreas.dil...@intel.com> wrote:
On 2016/01/12, 12:50, "lustre-discuss on behalf of Bob Ball"
<lustre-discuss-boun...@lists.lustre.org on behalf of b...@umich.edu>
wrote:
I have a zfs OST that I need to drain and re-create. This is lustre
2.7.x. In the past, with ldiskfs OST, I did this a number of times
following critical failures, and came up with the following items to be
replaced on the file system once the mkfs.lustre had been run, where the
new OST was mounted ldiskfs at /mnt/ost and the previous files had been
saved off in a backup_directory. This is also mostly documented in the
Lustre user manual.
cd backup_directory
cp -fv mountdata /mnt/ost/CONFIGS
cp last_rcvd /mnt/ost
mkdir -p /mnt/ost/O/0
chmod 700 /mnt/ost/O
chmod 700 /mnt/ost/O/0
cp -fv LAST_ID /mnt/ost/O/0
Now, with the zfs OST, mounted as type zfs, I see this file structure
drwxr-xr-x 2 root root 2 Dec 31 1969 CONFIGS
-rw-rw-rw- 1 root root 2 Dec 31 1969 fld
-rw-r--r-- 1 root root 0 Dec 31 1969 health_check
-rw-r--r-- 1 root root 8576 Dec 31 1969 last_rcvd
drw-r--r-- 2 root root 2 Dec 31 1969 LFSCK
-rw-r--r-- 1 root root 64 Dec 31 1969 lfsck_bookmark
-rw-r--r-- 1 root root 256 Dec 31 1969 lfsck_layout
drwxr-xr-x 1 root root 2 Dec 31 1969 O
drwxr-xr-x 1 root root 2 Dec 31 1969 oi.0
... with lots more oi.N
[root@umdist10 ~]# ll /mnt/temp/CONFIGS
total 258
-rw-r--r-- 1 root root 0 Dec 31 1969 params
-rw-r--r-- 1 root root 12104 Dec 31 1969 T3test-client
-rw-r--r-- 1 root root 8880 Dec 31 1969 T3test-OST0000
[root@umdist10 ~]# ll /mnt/temp/O
total 17
drwxr-xr-x 1 root root 2 Dec 31 1969 0
[root@umdist10 ~]# ll /mnt/temp/O/0
total 98784
drwxr-xr-x 1 root root 2 Dec 31 1969 d0
... 31 more dN directories
So, specifically, there is no LAST_ID file, and the CONFIGS directory
does not contain a "mountdata" file.
My questions is, do I just not worry about those 2 files, copy back the
last_rcvd file, remount the re-created OST, and continue on with life?
Or, are there other files from this zfs directory structure that should
also be saved and put back once the zpool is re-created?
For ZFS, the "mountdata" equivalent is stored as ZFS "lustre.*" dataset
parameters that can be read and modified via "zfs" in userspace. That
said, if you haven't changed anything from defaults they should be
recreated by mkfs.lustre the same way again.
For replacing the OST, you should use the "--replace" option in addition
to specifying "--index=N", so that it doesn't try to re-register with the
MGS as a new OST.
Copying "last_rcvd" isn't strictly needed, but doesn't hurt either.
As for LAST_ID, it seems that this object is present on ZFS OSTs, but
doesn't have a name. It is only looked up by FID from the OI, though it
would make sense to also create this file in the namespace for reference.
With newer versions of Lustre (2.5.0 and later with the LU-14 fixes) the
LAST_ID file will be recreated based on info from the MDS, so this
shouldn't be an obstacle either.
Bob, I'd suggest backing up the available files as you normally would, but
AFAIK you shouldn't actually need any of them anymore. If there are gaps
in the user manual in this area, it would be good to file an LUDOC bug
and/or a patch to the manual to address the gaps.
Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel High Performance Data Division
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
