Re: [openstack-dev] [Swift] Increase Swift ring partition power
Am 02.12.13 17:10, schrieb Gregory Holt: > On Dec 2, 2013, at 9:48 AM, Christian Schwede > wrote: > >> That sounds great! Is someone already working on this (I know about >> the ongoing DiskFile refactoring) or even a blueprint available? > > There is https://blueprints.launchpad.net/swift/+spec/ring-doubling > though I'm uncertain how up to date it is. Thanks for the link! I read all the linked entries, reviews and patches and it seems all of us wanted to use a similar approach. David put it in a nutshell: > > We can consider this to be the yearly event in which we try to crack > the part_power problem. > I'm going to write some docs and tests for my tool and will link it as related project afterwards. Christian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Swift] Increase Swift ring partition power
On Dec 2, 2013, at 9:48 AM, Christian Schwede wrote: > That sounds great! Is someone already working on this (I know about the > ongoing DiskFile refactoring) or even a blueprint available? There is https://blueprints.launchpad.net/swift/+spec/ring-doubling though I'm uncertain how up to date it is. Everybody works on everything ;) but Peter Portante has been the point on the DiskFile refactoring and I have been for the SSync part. David Hadas will probably kick back in once we (Peter and I) get a bit further down the line on our parts. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Swift] Increase Swift ring partition power
Am 02.12.13 15:47, schrieb Gregory Holt: > Achieving this transparently is part of the ongoing plans, starting > with things like the DiskFile refactoring and SSync. The idea is to > isolate the direct disk access from other servers/tools, something > that (for instance) RSync has today. Once the isolation is there, it > should be fairly straightforward to have incoming requests for a > ring^20 partition look on the local disk in a directory structure > that was originally created for a ring^19 partition, or even vice > versa. Then, there will be no need to move data around just for a > ring-doubling or halving, and no down time to do so. That sounds great! Is someone already working on this (I know about the ongoing DiskFile refactoring) or even a blueprint available? I was aware of the idea about multiple rings on the same policy but not about support for rings with a modified partition power. > That said, if you want create a tool that allows such ring shifting > in the interim, it should work with smaller clusters that don't mind > downtime. I would prefer that it not become a core tool checked > directly into swift/python-swiftclient, just because of the plans > stated above that should one day make it obsolete. Yes, that makes a lot of sense. In fact the tool is already working; I think the best way is to enhance the docs and to list it as a related Swift project once I'm done with this. Christian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Swift] Increase Swift ring partition power
On Dec 2, 2013, at 5:19 AM, Christian Schwede wrote: > I'd like to discuss a way to increase the partition power of an existing > Swift cluster. Achieving this transparently is part of the ongoing plans, starting with things like the DiskFile refactoring and SSync. The idea is to isolate the direct disk access from other servers/tools, something that (for instance) RSync has today. Once the isolation is there, it should be fairly straightforward to have incoming requests for a ring^20 partition look on the local disk in a directory structure that was originally created for a ring^19 partition, or even vice versa. Then, there will be no need to move data around just for a ring-doubling or halving, and no down time to do so. That said, if you want create a tool that allows such ring shifting in the interim, it should work with smaller clusters that don't mind downtime. I would prefer that it not become a core tool checked directly into swift/python-swiftclient, just because of the plans stated above that should one day make it obsolete. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
