On 15/02/16 23:29, Kota TSUYUZAKI wrote:
Hello Mark,
AFAIK, a few reasons for that we still are in working progress for erasure code
+ geo replication.
and expect to survive a region outage...
With that I mind I did some experiments (Liberty swift) and it looks to me like
if you have:
- num_data_frags < num_nodes in (smallest) region
and:
- num_parity_frags = num_data_frags
then having a region fail does not result in service outage.
Good point but note that the PyECLib v1.0.7 (pinned to Kilo/Liberty stable)
still have a problem which cannot decode the original data when all feed
fragments are parity frags[1]. (i.e. if set
num_parity_frags = num_data frags and then, num_parity_frags comes into proxy
for GET request, it will fail at the decoding) The problem was already resolved
in the PyECLib/liberasurecode at master
branch and current swift master has the PyECLib>=1.0.7 dependencies so if you
thought to use the newest Swift, it might be not
a matter.
Ah right, in my testing I always took down my "1st" region...which will
have had data fragments therein. For interest I'll try to provoke a
situation where I have all parity ones to assemble (and see what happens).
In the Swift perspective, I think that we need more tests/discussion for geo
replication around write/read affinity[2] which is geo replication stuff in
Swift itself and performances.
For the write/read affinity, actually we didn't consider the affinity control
to simplify the implementation until EC landed into Swift master[3] so I think
it's time to make sure how we can use the
affinity control with EC but it's not done yet.
For the performance perspective, in my experiments, more parities causes quite
performance degradation[4]. To prevent the degradation, I am working for the
spec which makes duplicated copy from
data/parity fragments and spread them out into geo regions.
To sumurize, we've not done the work yet but we welcome to discuss and
contribute for EC + geo replication anytime, IMO.
Thanks,
Kota
1:
https://bitbucket.org/tsg-/liberasurecode/commits/a01b1818c874a65d1d1fb8f11ea441e9d3e18771
2:
http://docs.openstack.org/developer/swift/admin_guide.html#geographically-distributed-clusters
3:
http://docs.openstack.org/developer/swift/overview_erasure_code.html#region-support
4:
https://specs.openstack.org/openstack/swift-specs/specs/in_progress/global_ec_cluster.html
Excellent - thank you for a very comprehensive answer.
Regards
Mark
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
