Re: [Gluster-users] Usage monitoring per user

[JOHE (John Hearns)]

I rather like ageduIt probably does what you want.

But as Mohammad says you do have to traverse your filesystem.


https://www.chiark.greenend.org.uk/~sgtatham/agedu/

agedu: track down wasted disk space - chiark home 
page<https://www.chiark.greenend.org.uk/~sgtatham/agedu/>
www.chiark.greenend.org.uk
agedu. a Unix utility for tracking down wasted disk space Introduction. Suppose 
you're running low on disk space. You need to free some up, by finding 
something that's a waste of space and deleting it (or moving it to an archive 
medium).







From: gluster-users-boun...@gluster.org <gluster-users-boun...@gluster.org> on 
behalf of Alex Chekholko <a...@calicolabs.com>
Sent: 01 May 2018 18:45
To: mohammad kashif
Cc: gluster-users
Subject: Re: [Gluster-users] Usage monitoring per user

Hi,

There are several programs that will basically take the outputs of your scans 
and store the results in a database. If you size the database appropriately, 
then querying that database will be much quicker than querying the filesystem.  
But of course the results will be a little bit outdated.

One such project is robinhood. 
https://github.com/cea-hpc/robinhood/wiki<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcea-hpc%2Frobinhood%2Fwiki=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0=m5s1l7DvoKiXlAhSP8eGlu9lYjtlkIM5k3CTGt6zoH0%3D=0>

A simpler way might be to just have daily/weekly cron jobs that output text 
reports, without maintaining a separate database.

But there is no way to avoid doing a recursive POSIX tree traversal, since that 
is how you get your info out of your filesystem.

Regards,
Alex

On Tue, May 1, 2018 at 5:30 AM, mohammad kashif 
<kashif.a...@gmail.com<mailto:kashif.a...@gmail.com>> wrote:
Hi

Is there any easy way to find usage per user in Gluster? We have 300TB storage 
with almost 100 million files. Running du take too much time. Are people aware 
of any other tool which can be used to break up storage per user?

Thanks

Kashif

___
Gluster-users mailing list
Gluster-users@gluster.org<mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.gluster.org%2Fmailman%2Flistinfo%2Fgluster-users=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0=P%2Bkfpz8XaI7j7PM9%2FocnBkf0Wop%2BLZDAbgnvNpIcdd0%3D=0>

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Usage monitoring per user

[mohammad kashif]

Hi Alex, John

Thanks for confirming my suspicion that there is no getting away from POSIX
tree traversal . I was aware of age-du but not robinhood.

Cheers

Kashif

On Wed, May 2, 2018 at 8:57 AM, JOHE (John Hearns) <j...@novozymes.com>
wrote:

> I rather like ageduIt probably does what you want.
>
> But as Mohammad says you do have to traverse your filesystem.
>
>
> https://www.chiark.greenend.org.uk/~sgtatham/agedu/
> agedu: track down wasted disk space - chiark home page
> <https://www.chiark.greenend.org.uk/~sgtatham/agedu/>
> www.chiark.greenend.org.uk
> agedu. a Unix utility for tracking down wasted disk space Introduction.
> Suppose you're running low on disk space. You need to free some up, by
> finding something that's a waste of space and deleting it (or moving it to
> an archive medium).
>
>
>
>
>
> --
> *From:* gluster-users-boun...@gluster.org <gluster-users-bounces@
> gluster.org> on behalf of Alex Chekholko <a...@calicolabs.com>
> *Sent:* 01 May 2018 18:45
> *To:* mohammad kashif
> *Cc:* gluster-users
> *Subject:* Re: [Gluster-users] Usage monitoring per user
>
> Hi,
>
> There are several programs that will basically take the outputs of your
> scans and store the results in a database. If you size the database
> appropriately, then querying that database will be much quicker than
> querying the filesystem.  But of course the results will be a little bit
> outdated.
>
> One such project is robinhood. https://github.com/cea-hpc/robinhood/wiki
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcea-hpc%2Frobinhood%2Fwiki=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0=m5s1l7DvoKiXlAhSP8eGlu9lYjtlkIM5k3CTGt6zoH0%3D=0>
>
> A simpler way might be to just have daily/weekly cron jobs that output
> text reports, without maintaining a separate database.
>
> But there is no way to avoid doing a recursive POSIX tree traversal, since
> that is how you get your info out of your filesystem.
>
> Regards,
> Alex
>
> On Tue, May 1, 2018 at 5:30 AM, mohammad kashif <kashif.a...@gmail.com>
> wrote:
>
> Hi
>
> Is there any easy way to find usage per user in Gluster? We have 300TB
> storage with almost 100 million files. Running du take too much time. Are
> people aware of any other tool which can be used to break up storage per
> user?
>
> Thanks
>
> Kashif
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.gluster.org%2Fmailman%2Flistinfo%2Fgluster-users=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0=P%2Bkfpz8XaI7j7PM9%2FocnBkf0Wop%2BLZDAbgnvNpIcdd0%3D=0>
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Usage monitoring per user

[Alex Chekholko]

Hi,

There are several programs that will basically take the outputs of your
scans and store the results in a database. If you size the database
appropriately, then querying that database will be much quicker than
querying the filesystem.  But of course the results will be a little bit
outdated.

One such project is robinhood. https://github.com/cea-hpc/robinhood/wiki

A simpler way might be to just have daily/weekly cron jobs that output text
reports, without maintaining a separate database.

But there is no way to avoid doing a recursive POSIX tree traversal, since
that is how you get your info out of your filesystem.

Regards,
Alex

On Tue, May 1, 2018 at 5:30 AM, mohammad kashif 
wrote:

> Hi
>
> Is there any easy way to find usage per user in Gluster? We have 300TB
> storage with almost 100 million files. Running du take too much time. Are
> people aware of any other tool which can be used to break up storage per
> user?
>
> Thanks
>
> Kashif
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Usage monitoring per user

[mohammad kashif]

Hi

Is there any easy way to find usage per user in Gluster? We have 300TB
storage with almost 100 million files. Running du take too much time. Are
people aware of any other tool which can be used to break up storage per
user?

Thanks

Kashif
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users