Clemens Wolff created HADOOP-16005:
--------------------------------------
Summary: NativeAzureFileSystem does not support setXAttr
Key: HADOOP-16005
URL: https://issues.apache.org/jira/browse/HADOOP-16005
Project: Hadoop Common
Issue Type: Bug
Components: fs/azure
Reporter: Clemens Wolff
When interacting with Azure Blob Storage via the Hadoop FileSystem client, it's
currently (as of
[a8bbd81|https://github.com/apache/hadoop/commit/a8bbd818d5bc4762324bcdb7cf1fdd5c2f93891b])
not possible to set custom metadata attributes.
Here is a snippet that demonstrates the missing behavior (throws an
UnsupportedOperationException):
{code:java}
val blobAccount = "SET ME"
val blobKey = "SET ME"
val blobContainer = "SET ME"
val blobFile = "SET ME"
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.{FileSystem, Path}
val conf = new Configuration()
conf.set("fs.wasbs.impl", "org.apache.hadoop.fs.azure.NativeAzureFileSystem")
conf.set(s"fs.azure.account.key.$blobAccount.blob.core.windows.net", blobKey)
val path = new
Path(s"wasbs://$blobContainer@$blobAccount.blob.core.windows.net/$blobFile")
val fs = FileSystem.get(path, conf)
fs.setXAttr(path, "somekey", "somevalue".getBytes)
{code}
Looking at the code in hadoop-tools/hadoop-azure, NativeAzureFileSystem
inherits the default setXAttr from FileSystem which throws the
UnsupportedOperationException.
The underlying Azure Blob Storage service does support custom metadata
([service
docs|https://docs.microsoft.com/en-us/azure/storage/blobs/storage-properties-metadata])
as does the azure-storage SDK that's being used by NativeAzureFileSystem ([SDK
docs|http://javadox.com/com.microsoft.azure/azure-storage/2.0.0/com/microsoft/azure/storage/blob/CloudBlob.html#setMetadata(java.util.HashMap)]).
Is there another way that I should be setting custom metadata on Azure Blob
Storage files? Is there a specific reason why setXAttr hasn't been implemented
on NativeAzureFileSystem? If not, I can take a shot at implementing it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
