Re: [lustre-discuss] Correct procedure for OST replacment

2022-10-26 Thread Tung-Han Hsieh
Dear Redl, Robert, On Wed, Oct 26, 2022 at 02:37:12PM +, Redl, Robert wrote: > Dear Etienne, > > thanks a lot for the detailed explanation! I will try out the patch at the > next opportunity. > > @Tung-Han Hsieh: I think the issue that indices of old OST remain until > --writeconf is

Re: [lustre-discuss] Quota issue after OST removal

2022-10-26 Thread Daniel Szkola via lustre-discuss
I did show a 'lfs quota -g somegroup' in the original post and yes, each OST is at the limit that was originally allocated, especially after migrating the files off of the two OSTs before removal. However, I think you may be misreading the issue here. The total quota is 27T and all the files on

Re: [lustre-discuss] Quota issue after OST removal

2022-10-26 Thread Thomas Roth via lustre-discuss
Hi Daniel, isn't this expected: on your lustrefs-OST0001, usage seems to have hit the limit (perhaps if you do 'lfs quota -g somegroup...', it will show you by how many bytes). If one part of the distributed quota is exceeded, Lustre should report that with the * - although the total across

[lustre-discuss] Quota issue after OST removal

2022-10-26 Thread Daniel Szkola via lustre-discuss
Hello all, We recently removed an OSS/OST node that was spontaneously shutting down so hardware testing could be performed. I have no idea how long it will be out, so I followed the procedure for permanent removal. Since then space usage is being calculated correctly, but 'lfs quota' will show

Re: [lustre-discuss] Correct procedure for OST replacment

2022-10-26 Thread Redl, Robert
Dear Etienne, thanks a lot for the detailed explanation! I will try out the patch at the next opportunity. @Tung-Han Hsieh: I think the issue that indices of old OST remain until --writeconf is used is solved by the new command lctl del_ost or the older lctl llog_cancel. Both are removing

Re: [lustre-discuss] Correct procedure for OST replacment

2022-10-26 Thread Tung-Han Hsieh
Hello, Just an experience to share. If we follow the correct procedure to permanently remove an OST, the index of that OST still exists in MDT. The only way to remove that OST index from MDT is to run "tunefs.lustre --writeconf" to MDT (and also to all OSTs). That needs temporarily shutdown the

Re: [lustre-discuss] Correct procedure for OST replacment

2022-10-26 Thread Etienne Aujames via lustre-discuss
Hi, "mkfs.lustre --replace", is used to replace an existing OST in MGS configurations (CONFIGS/*-{client,MDT*}). It will read the existing configuration on the MGS for the given index, copy it locally. Then it will negotiate LAST_IDs (last object id for each sequence) with MDTs (the OST should

[lustre-discuss] OST replacement procedure

2022-10-26 Thread Thomas Roth via lustre-discuss
Hi all, about the correct procedure to replace an OST: I read the recent issues reported here by Robert Redl, the LU-15000 by Stephane and in particular his talk at LAD22: Why is it important to _not_ reuse old OST indices? Understandable if you want to remove the OST, but not replace it.