Thanks for all this, Andreas. Always appreciated.
bob
On 7/17/2017 12:00 AM, Dilger, Andreas wrote:
When you write "MGS", you really mean "MDS". The MGS would be the
place for this if you were changing the config to permanently
deactivate the OSTs via "lctl conf_param". To temporarily do this, the
commands should be run on the MDS via "lctl set_param". In most cases
the MDS and MGS are co-located, so the distinction is irrelevant, but
good to get it right for the record.
The problem of objects not being unlinked until after the MDS is
restarted has been fixed.
Also, with 2.9 and later it is possible to use "lctl set_param
osp.<OST>.create_count=0" to stop new file allocation on that OST
without blocking unlinked at all, which is best for emptying old OSTs,
rather than using "deactivate".
For marking the OSTs read-only, both of these solutions will not
prevent clients from modifying the OST filesystems, just from creating
new files (assuming all OSTs are set this way).
You might consider to try "mount -o remount,ro" on the MDT and OST
filesystems on the servers to see if this works (I haven't tested this
myself). The problem might be that this prevents new clients from
mounting.
It probably makes sense to add server-side read-only mounting as a
feature. Could you please file a ticket in Jira about this?
Cheers, Andreas
On Jul 16, 2017, at 09:16, Bob Ball <b...@umich.edu
<mailto:b...@umich.edu>> wrote:
I agree with Raj. Also, I have noted with Lustre 2.7, that the space
is not actually freed after re-activation of the OST, until the mgs
is restarted. I don't recall the reason for this, or know if this
was fixed in later Lustre versions.
Remember, this is done on the mgs, not on the clients. If you do it
on a client, the behavior is as you thought.
bob
On 7/16/2017 11:10 AM, Raj wrote:
No. Deactivating an OST will not allow to create new objects(file).
But, client can read AND modify an existing objects(append the
file). Also, it will not free any space from deleted objects until
the OST is activated again.
On Sun, Jul 16, 2017, 9:29 AM E.S. Rosenberg
<esr+lus...@mail.hebrew.edu <mailto:esr%2blus...@mail.hebrew.edu>>
wrote:
On Thu, Jul 13, 2017 at 5:49 AM, Bob Ball <b...@umich.edu
<mailto:b...@umich.edu>> wrote:
On the mgs/mgt do something like:
lctl --device <fsname>-OST0019-osc-MDT0000 deactivate
No further files will be assigned to that OST. Reverse with
"activate". Or reboot the mgs/mdt as this is not
persistent. "lctl dl" will tell you exactly what that
device name should be for you.
Doesn't that also disable reads from the OST though?
bob
On 7/12/2017 6:04 PM, Alexander I Kulyavtsev wrote:
You may find advise from Andreas on this list (also
attached below). I did not try setting fail_loc myself.
In 2.9 there is setting osp.*.max_create_count=0described
at LUDOC-305.
We used to set OST degraded as described in lustre manual.
It works most of the time but at some point I saw lustre
errors in logs for some ops. Sorry, I do not recall details.
I still not sure either of these approaches will work for
you: setting OST degraded or fail_loc will makes some osts
selected instead of others.
You may want to verify if these settings will trigger clean
error on user side (instead of blocking) when all OSTs are
degraded.
The other and also simpler approach would be to enable
lustre quota and set quota below used space for all users
(or groups).
Alex.
*From: *"Dilger, Andreas" <andreas.dil...@intel.com
<mailto:andreas.dil...@intel.com>>
*Subject: **Re: [lustre-discuss] lustre 2.5.3 ost not
draining*
*Date: *July 28, 2015 at 11:51:38 PM CDT
*Cc: *"lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>"
<lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>>
Setting it degraded means the MDS will avoid allocations
on that OST
unless there aren't enough OSTs to meet the request (e.g.
stripe_count =
-1), so it should work.
That is actually a very interesting workaround for this
problem, and it
will work for older versions of Lustre as well. It
doesn't disable the
OST completely, which is fine if you are doing space
balancing (and may
even be desirable to allow apps that need more bandwidth
for a widely
striped file), but it isn't good if you are trying to
empty the OST
completely to remove it.
It looks like another approach would be to mark the OST as
having no free
space using OBD_FAIL_OST_ENOINO (0x229) fault injection on
that OST:
lctl set_param fail_loc=0x229 fail_val=<ost_index>
This would cause the OST to return 0 free inodes from
OST_STATFS for the
specified OST index, and the MDT would skip this OST
completely. To
disable all of the OSTs on an OSS use <ost_index> = -1.
It isn't possible
to selectively disable a subset of OSTs using this
method. The
OBD_FAIL_OST_ENOINO fail_loc has been available since
Lustre 2.2, which
covers all of the 2.4+ versions that are affected by this
issue.
If this mechanism works for you (it should, as this
fail_loc is used
during regular testing) I'd be obliged if someone could
file an LUDOC bug
so the manual can be updated.
Cheers, Andreas
On Jul 12, 2017, at 4:20 PM, Riccardo Veraldi
<riccardo.vera...@cnaf.infn.it
<mailto:riccardo.vera...@cnaf.infn.it>> wrote:
Hello,
on one of my lustre FS I need to find a solution so that
users can still
access data on the FS but cannot write new files on it.
I have hundreds of clients accessing the FS so remounting
it ro is not
really easily feasible.
Is there an option on the OSS side to allow OSTs to be
accessed just to
read data and not to store new data ?
tunefs.lustre could do that ?
thank you
Rick
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org <mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
