On Thu, Apr 05, 2018 at 04:03:04PM -0000, john.calc...@gmail.com wrote: > Thanks for the quick response Andrew - > > > The closet analog is AtmosUtils.putBlob which retries on > > KeyAlreadyExistsException after removing. Generally the jclouds > > portable abstraction tries to make all blobstores act the same and uses > > the native behavior for the providers. Which blobstore has similar > > behavior? I am not sure how we should handle this for almost > > S3-compatible implementations like Hitachi. > > So, clarifying - you're saying if it gets a KeyAlreadyExistsException, it > then deletes the key and retries the put? That seems a bit harsh - what if > you're building a distributed system on top of jclouds and you have two > cluster nodes racing to put the same key? Would it not be better to at least > test the metadata to see if you're trying to overwrite the same data and just > silently return ok?
Agreed that this is racy and something https://issues.apache.org/jira/browse/JCLOUDS-1111 unsuccessfully tried to address through a newer header that not all implementations support. Atmos does not return an ETag so we cannot check the same content, although ETag checking does not always work on S3, for example with multi-part uploads. Unfortunately Atmos is odd for a number of reasons and delete and retry was the best workaround at the time, especially for a low-popularity provider. Some blobstores like Ceph can address this issue with conditional PUT but this is not supported elsewhere. -- Andrew Gaul http://gaul.org/
