[Openstack] [SWIFT] Account reaping - gradually

2013-07-15 Thread Leandro Reox
Hi guys,

I was wondering if there any plans or maybe already exist a feature or a
flag for the account reaping process to make it gradually. We had one case
with an account with millions of files, that was plain deleted, an that
triggers the deletion of every object, then containers to finally delete
the account. This process added a LOT of load on the cluster causing bad
performance issues. Maybe we can add a flag like reaping_progress = x% or
something like that, maybe that hits recon to get the progress and when 5%
its done, move on with the next 5% and so on. Would be great to have
something like this without manual intervention.

Cheers
Lean
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [SWIFT] Account reaping - gradually

2013-07-15 Thread Samuel Merritt

On 7/15/13 6:52 AM, Leandro Reox wrote:

Hi guys,

I was wondering if there any plans or maybe already exist a feature or a
flag for the account reaping process to make it gradually. We had one
case with an account with millions of files, that was plain deleted, an
that triggers the deletion of every object, then containers to finally
delete the account. This process added a LOT of load on the cluster
causing bad performance issues. Maybe we can add a flag like
reaping_progress = x% or something like that, maybe that hits recon to
get the progress and when 5% its done, move on with the next 5% and so
on. Would be great to have something like this without manual intervention.


There is a concurrency setting (imaginatively named concurrency) in 
the account-reaper config that controls the maximum number of concurrent 
DELETE requests. Default is 25; sounds like you might want to lower it.


Note that the value should be a perfect square (so 1, 4, 9, etc.); if 
not, the reaper will still work, but the number will effectively be 
rounded up to the next perfect square (so 20 - 25, for example).



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp