In a standard swift deployment, the proxy server is running behind a load 
balancer and/or an SSL terminator. At SwiftStack, we discovered an issue that 
may arise from some config parameters in this layer, and we'd like to share it 
with other swift deployers.

Symptom:

Users updating metadata (ie POST) on larger objects get 503 error responses. 
However, there are no error responses logged by swift.

Cause:

Since POSTs are implemented, by default, as a server-side copy in swift and 
there is no traffic between the user and swift during the server-side copy, the 
LB or SSL terminator times out before the operation is done.

Solution:

Two options:

1) Raise the timeout in the LB/SSL terminator config. For example, with pound 
change the "TimeOut" for the swift backend. pound defaults to 15 seconds. The 
appropriate value is however log it takes to do a server side copy of your 
largest object. If you have a 1gbps network, it will take about 160 seconds to 
copy a 5GB object ((8*5*2**30)/((2**30)/4) -- the divide by 4 is because the 
1gbps link is used to read one stream (the original) and write the new copy (3 
replicas)).

2) Change the behavior of POSTs to not do a server-side copy. This will make 
POSTs faster, but it will prevent all metadata values from being updated 
(notably, Content-Type will not be able to be modified with a POST). Also, this 
will not make the issue go away with user-initiated server-side copies.

I would recommend the first solution, unless your workload makes heavy use of 
POSTs.

Hoep this helps.

--John


Attachment: smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Reply via email to