Re: [lustre-discuss] Lustre recycle bin

[Degremont, Aurelien via lustre-discuss]

Hi François

[root@server1 ~] rm: cannot remove ‘Logs: Cannot send after transport endpoint 
shutdown
[root@server1 ~] mv: cannot move /test/lustre/structure1 to 
‘/test/lustre/structure2’: Input/output error

These 2 error messages are typical from a client eviction issue. Your client 
was evicted (likely from the MDT) and you should expect ESHUTDOWN or EIO in 
that situation. Look at the kernel logs (dmesg) from both client and server to 
try understand. The client usually reconnect quickly when this happens and 
following access should not return error.

Aurélien

De : lustre-discuss  au nom de 
"Cloete, F. (Francois) via lustre-discuss" 
Répondre à : "Cloete, F. (Francois)" 
Date : jeudi 20 octobre 2022 à 08:48
À : "Spitz, Cory James" , Alastair Basden 

Cc : "lustre-discuss@lists.lustre.org" 
Objet : RE: [EXTERNAL][lustre-discuss] Lustre recycle bin


CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.


Hi Cory,
They are both running the same versions on client and mgs server.

lfs 2.12.8_6_g5457c37

Not sure if this could be related but our lustre environment started behaving 
starngly 2/3 weeks ago.

Also seeing the below when doing rsync of folders to new destinations.
failed: Cannot send after transport endpoint shutdown (108)


[root@server1 ~] rm: cannot remove ‘Logs: Cannot send after transport endpoint 
shutdown
[root@server1 ~] mv: cannot move /test/lustre/structure1 to 
‘/test/lustre/structure2’: Input/output error

[root@server1 ~] ~] ll
[root@server1 ~] ls: cannot open directory .: Cannot send after transport 
endpoint shutdown

Regards

From: Spitz, Cory James 
Sent: Monday, 17 October 2022 20:56
To: Cloete, F. (Francois) ; Alastair Basden 

Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin


You don't often get email from cory.sp...@hpe.com<mailto:cory.sp...@hpe.com>. 
Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>

CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
What version(s) are you using?  Do you have an old client and a new-ish server?

Very old client versions will disagree with the MDSes about how to clean up 
objects, resulting in orphans.

-Cory

On 10/17/22, 3:44 AM, "lustre-discuss" 
mailto:lustre-discuss-boun...@lists.lustre.org>>
 wrote:

Thank-you!

-Original Message-
From: Alastair Basden mailto:a.g.bas...@durham.ac.uk>>
Sent: Monday, 17 October 2022 10:13
To: Cloete, F. (Francois) 
mailto:francois...@nedbank.co.za>>
Cc: Andreas Dilger mailto:adil...@whamcloud.com>>; 
lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
Subject: Re: [lustre-discuss] Lustre recycle bin

[You don't often get email from 
a.g.bas...@durham.ac.uk<mailto:a.g.bas...@durham.ac.uk>. Learn why this is 
important at https://aka.ms/LearnAboutSenderIdentification   ]


CAUTION - EXTERNAL SENDER -  Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)


Hi Francois,

We had something similar a few months back - I suspect a bug somewhere.

Basically files weren't getting removed from the OST.  Eventually, we mounted 
as ext, and removed them manually, I think.

A reboot of the file system meant that rm operations then proceeded correctly 
after that.

Cheers,
Alastair.

On Mon, 17 Oct 2022, Cloete, F. (Francois) via lustre-discuss wrote:

> [EXTERNAL EMAIL]
> Hi Andreas,
> Our OSTs still display high file-system usage after removing folders.
>
> Are there any commands that could be run to confirm if the allocated space 
> which was used by those files have been released successfully ?
>
> Thanks
> Francois
>
> From: Andreas Dilger mailto:adil...@whamcloud.com>>
> Sent: Saturday, 15 October 2022 00:20
> To: Cloete, F. (Francois) 
> mailto:francois...@nedbank.co.za>>
> Cc: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
> Subject: Re: [lustre-discuss] Lustre recycle bin
>
> You don't often get email from
> adil...@whamcloud.com<mailto:adil...@whamcloud.com<mailto:adil...@whamcloud.com%3cmailto:adil...@whamcloud.com>>.
>  Learn why this is
> important<https://aka.ms/LearnAboutSenderIdentification 
> <https://aka.ms/LearnAboutSenderIdentification%20> >
> CAUTION - EXTERNAL SENDER - Please be careful

Re: [lustre-discuss] Lustre recycle bin

[Cloete, F. (Francois) via lustre-discuss]

Hi Cory,
They are both running the same versions on client and mgs server.

lfs 2.12.8_6_g5457c37

Not sure if this could be related but our lustre environment started behaving 
starngly 2/3 weeks ago.

Also seeing the below when doing rsync of folders to new destinations.
failed: Cannot send after transport endpoint shutdown (108)


[root@server1 ~] rm: cannot remove 'Logs: Cannot send after transport endpoint 
shutdown
[root@server1 ~] mv: cannot move /test/lustre/structure1 to 
'/test/lustre/structure2': Input/output error

[root@server1 ~] ~] ll
[root@server1 ~] ls: cannot open directory .: Cannot send after transport 
endpoint shutdown

Regards

From: Spitz, Cory James 
Sent: Monday, 17 October 2022 20:56
To: Cloete, F. (Francois) ; Alastair Basden 

Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

You don't often get email from cory.sp...@hpe.com<mailto:cory.sp...@hpe.com>. 
Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
What version(s) are you using?  Do you have an old client and a new-ish server?

Very old client versions will disagree with the MDSes about how to clean up 
objects, resulting in orphans.

-Cory

On 10/17/22, 3:44 AM, "lustre-discuss" 
mailto:lustre-discuss-boun...@lists.lustre.org>>
 wrote:

Thank-you!

-Original Message-
From: Alastair Basden mailto:a.g.bas...@durham.ac.uk>>
Sent: Monday, 17 October 2022 10:13
To: Cloete, F. (Francois) 
mailto:francois...@nedbank.co.za>>
Cc: Andreas Dilger mailto:adil...@whamcloud.com>>; 
lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
Subject: Re: [lustre-discuss] Lustre recycle bin

[You don't often get email from 
a.g.bas...@durham.ac.uk<mailto:a.g.bas...@durham.ac.uk>. Learn why this is 
important at https://aka.ms/LearnAboutSenderIdentification   ]


CAUTION - EXTERNAL SENDER -  Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)


Hi Francois,

We had something similar a few months back - I suspect a bug somewhere.

Basically files weren't getting removed from the OST.  Eventually, we mounted 
as ext, and removed them manually, I think.

A reboot of the file system meant that rm operations then proceeded correctly 
after that.

Cheers,
Alastair.

On Mon, 17 Oct 2022, Cloete, F. (Francois) via lustre-discuss wrote:

> [EXTERNAL EMAIL]
> Hi Andreas,
> Our OSTs still display high file-system usage after removing folders.
>
> Are there any commands that could be run to confirm if the allocated space 
> which was used by those files have been released successfully ?
>
> Thanks
> Francois
>
> From: Andreas Dilger mailto:adil...@whamcloud.com>>
> Sent: Saturday, 15 October 2022 00:20
> To: Cloete, F. (Francois) 
> mailto:francois...@nedbank.co.za>>
> Cc: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
> Subject: Re: [lustre-discuss] Lustre recycle bin
>
> You don't often get email from
> adil...@whamcloud.com<mailto:adil...@whamcloud.com<mailto:adil...@whamcloud.com%3cmailto:adil...@whamcloud.com>>.
>  Learn why this is
> important<https://aka.ms/LearnAboutSenderIdentification 
> <https://aka.ms/LearnAboutSenderIdentification%20> >
> CAUTION - EXTERNAL SENDER - Please be careful when opening links and
> attachments. Nedbank - IT Information Security Department (ISD) There isn't a 
> recycle bin, but filenames are deleted from the filesystem quickly and the 
> data objects are deleted in the background asynchronously (with transactions 
> to prevent the space being leaked).  If there are a lot of files this may 
> take some time, rebooting will not speed it up.
>
>
> On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
> mailto:lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org%3cmailto:lustre-discuss@lists.lustre.org>>>
>  wrote:
>
> Hi Community,
> Is anyone aware of a recycle bin parameter for Lustre?
>
> Just deleted a whole lot of files but for some reason the space is not 
> getting cleared.
>
> Server rebooted, file-system un-mounted etc.
>
> Thanks
>
> 
> Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
> notice:
> This email, including any attachments (email), c

Re: [lustre-discuss] Lustre recycle bin

[Spitz, Cory James via lustre-discuss]

What version(s) are you using?  Do you have an old client and a new-ish server?

Very old client versions will disagree with the MDSes about how to clean up 
objects, resulting in orphans.

-Cory

On 10/17/22, 3:44 AM, "lustre-discuss" 
 wrote:

Thank-you!

-Original Message-
From: Alastair Basden 
Sent: Monday, 17 October 2022 10:13
To: Cloete, F. (Francois) 
Cc: Andreas Dilger ; lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

[You don't often get email from a.g.bas...@durham.ac.uk. Learn why this is 
important at 
https://aka.ms/LearnAboutSenderIdentification<https://aka.ms/LearnAboutSenderIdentification>
   ]


CAUTION - EXTERNAL SENDER -  Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)


Hi Francois,

We had something similar a few months back - I suspect a bug somewhere.

Basically files weren't getting removed from the OST.  Eventually, we mounted 
as ext, and removed them manually, I think.

A reboot of the file system meant that rm operations then proceeded correctly 
after that.

Cheers,
Alastair.

On Mon, 17 Oct 2022, Cloete, F. (Francois) via lustre-discuss wrote:

> [EXTERNAL EMAIL]
> Hi Andreas,
> Our OSTs still display high file-system usage after removing folders.
>
> Are there any commands that could be run to confirm if the allocated space 
> which was used by those files have been released successfully ?
>
> Thanks
> Francois
>
> From: Andreas Dilger 
> Sent: Saturday, 15 October 2022 00:20
> To: Cloete, F. (Francois) 
> Cc: lustre-discuss@lists.lustre.org
> Subject: Re: [lustre-discuss] Lustre recycle bin
>
> You don't often get email from
> adil...@whamcloud.com<mailto:adil...@whamcloud.com>. Learn why this is
> important<https://aka.ms/LearnAboutSenderIdentification  >
> CAUTION - EXTERNAL SENDER - Please be careful when opening links and
> attachments. Nedbank - IT Information Security Department (ISD) There isn't a 
> recycle bin, but filenames are deleted from the filesystem quickly and the 
> data objects are deleted in the background asynchronously (with transactions 
> to prevent the space being leaked).  If there are a lot of files this may 
> take some time, rebooting will not speed it up.
>
>
> On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
> mailto:lustre-discuss@lists.lustre.org>> 
> wrote:
>
> Hi Community,
> Is anyone aware of a recycle bin parameter for Lustre?
>
> Just deleted a whole lot of files but for some reason the space is not 
> getting cleared.
>
> Server rebooted, file-system un-mounted etc.
>
> Thanks
>
> 
> Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
> notice:
> This email, including any attachments (email), contains information that is 
> confidential and is meant only for intended recipients. You may not share or 
> copy the email or any part of it, unless the sender has specifically allowed 
> you to do so. If you are not an intended recipient, please delete the email 
> permanently and let Nedbank know that you have deleted it by replying to the 
> sender or calling the Nedbank Contact Centre on 0860 555 111.
> This email is not confirmation of a transaction or a Nedbank statement and is 
> not offering or inviting anyone to take up any financial products or 
> services, unless the content specifically indicates that it does so. Nedbank 
> will not be liable for any errors or omissions in this email. The views and 
> opinions are those of the author and not necessarily those of Nedbank.
> The names of the Nedbank Board of Directors and Company Secretary are 
> available here: 
> http://www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>
>   <http://www.nedbank.co.za/terms/DirectorsNedbank.htm  >. Nedbank Ltd Reg No 
> 1951/09/06.
> 
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org
> >
> https://eur01.safelinks.protection.outlook.com/?url=http://lists<https://eur01.safelinks.protection.outlook.com/?url=http://lists>
> .lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.orgdata=05%7C0
> 1%7CFrancoisClo%40Nedbank.co.za%7Caccd70e00cb84df3418608dab017807f%7C0
> b1d23d810d140938cb7fd0bb32f81e1%7C0%7C0%7C6380159122763676

Re: [lustre-discuss] Lustre recycle bin

[Cloete, F. (Francois) via lustre-discuss]

Thank-you! 

-Original Message-
From: Alastair Basden  
Sent: Monday, 17 October 2022 10:13
To: Cloete, F. (Francois) 
Cc: Andreas Dilger ; lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

[You don't often get email from a.g.bas...@durham.ac.uk. Learn why this is 
important at https://aka.ms/LearnAboutSenderIdentification ]


CAUTION - EXTERNAL SENDER -  Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)


Hi Francois,

We had something similar a few months back - I suspect a bug somewhere.

Basically files weren't getting removed from the OST.  Eventually, we mounted 
as ext, and removed them manually, I think.

A reboot of the file system meant that rm operations then proceeded correctly 
after that.

Cheers,
Alastair.

On Mon, 17 Oct 2022, Cloete, F. (Francois) via lustre-discuss wrote:

> [EXTERNAL EMAIL]
> Hi Andreas,
> Our OSTs still display high file-system usage after removing folders.
>
> Are there any commands that could be run to confirm if the allocated space 
> which was used by those files have been released successfully ?
>
> Thanks
> Francois
>
> From: Andreas Dilger 
> Sent: Saturday, 15 October 2022 00:20
> To: Cloete, F. (Francois) 
> Cc: lustre-discuss@lists.lustre.org
> Subject: Re: [lustre-discuss] Lustre recycle bin
>
> You don't often get email from 
> adil...@whamcloud.com<mailto:adil...@whamcloud.com>. Learn why this is 
> important<https://aka.ms/LearnAboutSenderIdentification>
> CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
> attachments. Nedbank - IT Information Security Department (ISD) There isn't a 
> recycle bin, but filenames are deleted from the filesystem quickly and the 
> data objects are deleted in the background asynchronously (with transactions 
> to prevent the space being leaked).  If there are a lot of files this may 
> take some time, rebooting will not speed it up.
>
>
> On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
> mailto:lustre-discuss@lists.lustre.org>> 
> wrote:
>
> Hi Community,
> Is anyone aware of a recycle bin parameter for Lustre?
>
> Just deleted a whole lot of files but for some reason the space is not 
> getting cleared.
>
> Server rebooted, file-system un-mounted etc.
>
> Thanks
>
> 
> Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
> notice:
> This email, including any attachments (email), contains information that is 
> confidential and is meant only for intended recipients. You may not share or 
> copy the email or any part of it, unless the sender has specifically allowed 
> you to do so. If you are not an intended recipient, please delete the email 
> permanently and let Nedbank know that you have deleted it by replying to the 
> sender or calling the Nedbank Contact Centre on 0860 555 111.
> This email is not confirmation of a transaction or a Nedbank statement and is 
> not offering or inviting anyone to take up any financial products or 
> services, unless the content specifically indicates that it does so. Nedbank 
> will not be liable for any errors or omissions in this email. The views and 
> opinions are those of the author and not necessarily those of Nedbank.
> The names of the Nedbank Board of Directors and Company Secretary are 
> available here: 
> www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>.
>  Nedbank Ltd Reg No 1951/09/06.
> 
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org
> >
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists
> .lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.orgdata=05%7C0
> 1%7CFrancoisClo%40Nedbank.co.za%7Caccd70e00cb84df3418608dab017807f%7C0
> b1d23d810d140938cb7fd0bb32f81e1%7C0%7C0%7C638015912276367615%7CUnknown
> %7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> XVCI6Mn0%3D%7C3000%7C%7C%7Csdata=f4pQ6fAnpWfLFoihgUUGBNXH65%2B9Yc
> Wgotw5E7UuUgU%3Dreserved=0
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud
>
>
>
>
>
>
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Alastair Basden via lustre-discuss]

Hi Francois,

We had something similar a few months back - I suspect a bug somewhere.

Basically files weren't getting removed from the OST.  Eventually, we 
mounted as ext, and removed them manually, I think.


A reboot of the file system meant that rm operations then proceeded 
correctly after that.


Cheers,
Alastair.

On Mon, 17 Oct 2022, Cloete, F. (Francois) via lustre-discuss wrote:


[EXTERNAL EMAIL]
Hi Andreas,
Our OSTs still display high file-system usage after removing folders.

Are there any commands that could be run to confirm if the allocated space 
which was used by those files have been released successfully ?

Thanks
Francois

From: Andreas Dilger 
Sent: Saturday, 15 October 2022 00:20
To: Cloete, F. (Francois) 
Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

You don't often get email from adil...@whamcloud.com<mailto:adil...@whamcloud.com>. 
Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
There isn't a recycle bin, but filenames are deleted from the filesystem 
quickly and the data objects are deleted in the background asynchronously (with 
transactions to prevent the space being leaked).  If there are a lot of files 
this may take some time, rebooting will not speed it up.


On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
mailto:lustre-discuss@lists.lustre.org>> wrote:

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks


Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:
This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.
This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.
The names of the Nedbank Board of Directors and Company Secretary are available here: 
www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>.
 Nedbank Ltd Reg No 1951/09/06.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Cloete, F. (Francois) via lustre-discuss]

Thanks Hans!

From: lustre-discuss  On Behalf Of 
Hans Henrik Happe via lustre-discuss
Sent: Monday, 17 October 2022 08:44
To: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
Hi Francois,

Perhaps, you could try lfsck with the -o option to find orphan objects (man 
lctl-lfsck-start).

Cheers,
Hans Henrik
On 17.10.2022 08.01, Cloete, F. (Francois) via lustre-discuss wrote:
Hi Andreas,
Our OSTs still display high file-system usage after removing folders.

Are there any commands that could be run to confirm if the allocated space 
which was used by those files have been released successfully ?

Thanks
Francois

From: Andreas Dilger <mailto:adil...@whamcloud.com>
Sent: Saturday, 15 October 2022 00:20
To: Cloete, F. (Francois) 
<mailto:francois...@nedbank.co.za>
Cc: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
Subject: Re: [lustre-discuss] Lustre recycle bin

You don't often get email from 
adil...@whamcloud.com<mailto:adil...@whamcloud.com>. Learn why this is 
important<https://aka.ms/LearnAboutSenderIdentification>
CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
There isn't a recycle bin, but filenames are deleted from the filesystem 
quickly and the data objects are deleted in the background asynchronously (with 
transactions to prevent the space being leaked).  If there are a lot of files 
this may take some time, rebooting will not speed it up.



On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
mailto:lustre-discuss@lists.lustre.org>> wrote:

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks


Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:
This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.
This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.
The names of the Nedbank Board of Directors and Company Secretary are available 
here: 
www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>.
 Nedbank Ltd Reg No 1951/09/06.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org=05%7C01%7CFrancoisClo%40Nedbank.co.za%7Cbf6ba935ffb74b0234eb08dab00b1b73%7C0b1d23d810d140938cb7fd0bb32f81e1%7C0%7C0%7C638015859037895576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=XaPVAIaNV1A86Q6oYAHHCAj%2FgS7MSwfpFdkUYqMK63M%3D=0>

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud









___

lustre-discuss mailing list

lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>

http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org=05%7C01%7CFrancoisClo%40Nedbank.co.za%7Cbf6ba935ffb74b0234eb08dab00b1b73%7C0b1d23d810d140938cb7fd0bb32f81e1%7C0%7C0%7C638015859037895576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=XaPVAIaNV1A86Q6oYAHHCAj%2FgS7MSwfpFdkUYqMK63M%3D=0>

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Hans Henrik Happe via lustre-discuss]

Hi Francois,

Perhaps, you could try lfsck with the -o option to find orphan objects 
(man lctl-lfsck-start).


Cheers,
Hans Henrik

On 17.10.2022 08.01, Cloete, F. (Francois) via lustre-discuss wrote:


Hi Andreas,

Our OSTs still display high file-system usage after removing folders.

Are there any commands that could be run to confirm if the allocated 
space which was used by those files have been released successfully ?


Thanks

Francois

*From:*Andreas Dilger 
*Sent:* Saturday, 15 October 2022 00:20
*To:* Cloete, F. (Francois) 
*Cc:* lustre-discuss@lists.lustre.org
*Subject:* Re: [lustre-discuss] Lustre recycle bin




You don't often get email from adil...@whamcloud.com. Learn why this 
is important <https://aka.ms/LearnAboutSenderIdentification>




*CAUTION - EXTERNAL SENDER -***Please be careful when opening links 
and attachments. Nedbank - IT Information Security Department (ISD)


There isn't a recycle bin, but filenames are deleted from the 
filesystem quickly and the data objects are deleted in the background 
asynchronously (with transactions to prevent the space being leaked). 
 If there are a lot of files this may take some time, rebooting will 
not speed it up.




On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via
lustre-discuss  wrote:

Hi Community,

Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is
not getting cleared.

Server rebooted, file-system un-mounted etc.

Thanks


*Nedbank Limited group of companies (Nedbank) disclaimer and
confidentiality notice:*

This email, including any attachments (email), contains
information that is confidential and is meant only for intended
recipients. You may not share or copy the email or any part of it,
unless the sender has specifically allowed you to do so. If you
are not an intended recipient, please delete the email permanently
and let Nedbank know that you have deleted it by replying to the
sender or calling the Nedbank Contact Centre on 0860 555 111.

This email is not confirmation of a transaction or a Nedbank
statement and is not offering or inviting anyone to take up any
financial products or services, unless the content specifically
indicates that it does so. Nedbank will not be liable for any
errors or omissions in this email. The views and opinions are
those of the author and not necessarily those of Nedbank.

The names of the Nedbank Board of Directors and Company Secretary
are available here: www.nedbank.co.za/terms/DirectorsNedbank.htm
<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>. Nedbank Ltd
Reg No 1951/09/06.


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas

--

Andreas Dilger

Lustre Principal Architect

Whamcloud


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Cloete, F. (Francois) via lustre-discuss]

Hi Andreas,
Our OSTs still display high file-system usage after removing folders.

Are there any commands that could be run to confirm if the allocated space 
which was used by those files have been released successfully ?

Thanks
Francois

From: Andreas Dilger 
Sent: Saturday, 15 October 2022 00:20
To: Cloete, F. (Francois) 
Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Lustre recycle bin

You don't often get email from 
adil...@whamcloud.com<mailto:adil...@whamcloud.com>. Learn why this is 
important<https://aka.ms/LearnAboutSenderIdentification>
CAUTION - EXTERNAL SENDER - Please be careful when opening links and 
attachments. Nedbank - IT Information Security Department (ISD)
There isn't a recycle bin, but filenames are deleted from the filesystem 
quickly and the data objects are deleted in the background asynchronously (with 
transactions to prevent the space being leaked).  If there are a lot of files 
this may take some time, rebooting will not speed it up.


On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
mailto:lustre-discuss@lists.lustre.org>> wrote:

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks


Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:
This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.
This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.
The names of the Nedbank Board of Directors and Company Secretary are available 
here: 
www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>.
 Nedbank Ltd Reg No 1951/09/06.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Andreas Dilger via lustre-discuss]

There isn't a recycle bin, but filenames are deleted from the filesystem 
quickly and the data objects are deleted in the background asynchronously (with 
transactions to prevent the space being leaked).  If there are a lot of files 
this may take some time, rebooting will not speed it up.

On Oct 14, 2022, at 10:00, Cloete, F. (Francois) via lustre-discuss 
mailto:lustre-discuss@lists.lustre.org>> wrote:

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks



Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:

This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.

This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.

The names of the Nedbank Board of Directors and Company Secretary are available 
here: 
www.nedbank.co.za/terms/DirectorsNedbank.htm.
 Nedbank Ltd Reg No 1951/09/06.


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre recycle bin

[Andreas Krause via lustre-discuss]

Hi,
There isn't a recycle bin per-se.
File/directory deletion like 'rm -r ...' trying to delete a complex directory 
structure can take a while, depending on the MDS utilization, MDT performance 
characteristics and DNE setup if used.
More so if it is primarily smaller files.
It's worth a try using 'lunlink' when deleting a large number of objects and 
the capacity needs to be vacated as quickly as possible.

Thx,
Andreas


From: lustre-discuss  On Behalf Of 
Cloete, F. (Francois) via lustre-discuss
Sent: Friday, October 14, 2022 12:01 PM
To: lustre-discuss@lists.lustre.org
Subject: [lustre-discuss] Lustre recycle bin

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks



Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:

This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.

This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.

The names of the Nedbank Board of Directors and Company Secretary are available 
here: 
www.nedbank.co.za/terms/DirectorsNedbank.htm<http://www.nedbank.co.za/terms/DirectorsNedbank.htm>.
 Nedbank Ltd Reg No 1951/09/06.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Lustre recycle bin

[Cloete, F. (Francois) via lustre-discuss]

Hi Community,
Is anyone aware of a recycle bin parameter for Lustre?

Just deleted a whole lot of files but for some reason the space is not getting 
cleared.

Server rebooted, file-system un-mounted etc.

Thanks



Nedbank Limited group of companies (Nedbank) disclaimer and confidentiality 
notice:

This email, including any attachments (email), contains information that is 
confidential and is meant only for intended recipients. You may not share or 
copy the email or any part of it, unless the sender has specifically allowed 
you to do so. If you are not an intended recipient, please delete the email 
permanently and let Nedbank know that you have deleted it by replying to the 
sender or calling the Nedbank Contact Centre on 0860 555 111.

This email is not confirmation of a transaction or a Nedbank statement and is 
not offering or inviting anyone to take up any financial products or services, 
unless the content specifically indicates that it does so. Nedbank will not be 
liable for any errors or omissions in this email. The views and opinions are 
those of the author and not necessarily those of Nedbank.

The names of the Nedbank Board of Directors and Company Secretary are available 
here: www.nedbank.co.za/terms/DirectorsNedbank.htm. Nedbank Ltd Reg No 
1951/09/06.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org