c-w commented on a change in pull request #452: HADOOP-16005: Add XAttr support to WASB and ABFS URL: https://github.com/apache/hadoop/pull/452#discussion_r352266502
########## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java ########## @@ -247,6 +249,8 @@ private static final int DEFAULT_CONCURRENT_WRITES = 8; + private static final Charset METADATA_ENCODING = StandardCharsets.UTF_8; Review comment: The documentation for Azure Data Lake Storage Gen2 (backing `AzureBlobFileSystemStore`) states that x-ms-properties must be encoded in ISO-8859-1 (see [path/update#request-headers](https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/update#request-headers)). The documentation for Azure Blob Storage (backing `AzureNativeFileSystemStore`) however only states that x-ms-meta must follow the conventions for C# identifiers (see [set-blob-metadata#request-headers](https://docs.microsoft.com/en-us/rest/api/storageservices/set-blob-metadata#request-headers)) which may contain Unicode letters and characters (see [identifier-names](https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/inside-a-program/identifier-names)). As such I believe that the two classes should use different encodings. What am I missing? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
