Hi,
I'm working on OAK-8105 which is to update AzureDataStore to use the new
Azure v12 SDK instead of the deprecated v8 SDK, and may have run into a
snag where I could use some input from the team.
The main issue: Current cloud data store implementations (Azure and S3)
have the following behavior: When a client tries to write a binary that
already exists in blob storage, instead of writing the binary, the existing
binary has the last-modified time updated and a record for the existing
binary is returned as the result. The question is: What would be the
impact if we were unable to update the last-modified time in this situation?
Background: AzureDataStore currently allows authentication/authorization
to the Azure storage service two different ways. One is via an access key
- essentially a shared secret created by the storage service. The other
way is via a shared access signature, which can be generated via an API
call. Importantly we don't use "both" in a single instance - we use the
access key if it is provided, and otherwise use the shared access signature.
Azure's API does not allow modifying the last-modified property of a blob
directly. To do this up until now we have issued a service-side blob copy
instruction to copy the blob to itself, which has the effect of updating
the last-modified value.
However, with the new Azure SDK, based on my testing there are certain API
operations that you cannot perform when you authenticate with a shared
access signature. One of these actions you cannot perform is a
service-side blob copy. I am working with Microsoft directly to try to
find a workaround, but if my testing is correct we may not be able to
update the last-modified value in the situation of writing an already
existing binary, if a shared access signature is used to authenticate.
(It is possible this never worked with the old SDK either; I don't think
that particular behavior was ever tested using a shared access signature
before today.)
If we cannot find a workaround I see the following options:
- Don't update the last-modified value if we authenticate using a shared
access signature. (Or don't worry about it at all if it doesn't actually
matter - but I assume it does matter.)
- Don't allow authentication/authorization with shared access signatures
for AzureDataStore. (This would potentially break existing implementations
that are using this method to authenticate.)
Sorry for the long email, but I thought the full context was necessary.
Open to thoughts on this.
-MR
