[
https://issues.apache.org/jira/browse/HADOOP-17643?focusedWorklogId=606694&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-606694
]
ASF GitHub Bot logged work on HADOOP-17643:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jun/21 08:15
Start Date: 04/Jun/21 08:15
Worklog Time Spent: 10m
Work Description: anoopsjohn commented on a change in pull request #2972:
URL: https://github.com/apache/hadoop/pull/2972#discussion_r644923176
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
##########
@@ -1698,19 +1721,32 @@ private static void storeLinkAttribute(CloudBlobWrapper
blob,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
}
- private static String getLinkAttributeValue(CloudBlobWrapper blob)
+ private String getLinkAttributeValue(CloudBlobWrapper blob)
throws UnsupportedEncodingException {
- String encodedLinkTarget = getMetadataAttribute(blob,
+ String encodedLinkTarget = getMetadataAttribute(blob.getMetadata(),
LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
return decodeMetadataAttribute(encodedLinkTarget);
}
- private static boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
+ private boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
HashMap<String, String> metadata = blob.getMetadata();
- return null != metadata
- && (metadata.containsKey(IS_FOLDER_METADATA_KEY) || metadata
- .containsKey(OLD_IS_FOLDER_METADATA_KEY));
+ if (null != metadata) {
Review comment:
Reason why that was not done is to avoid extra call Map#get().
retrieveFolderAttribute() will be a very often called API. So I did not want to
make any extra calls as part of this change, specially in default case of
'fs.azure.blob.metadata.key.case.sensitive' false.
Also practically it may not possible but still its possible that a Map
contains a given Key but with value as null. What we used to check at this API
was/is just check whether key is present in Map or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 606694)
Time Spent: 1h (was: 50m)
> WASB : Make metadata checks case insensitive
> --------------------------------------------
>
> Key: HADOOP-17643
> URL: https://issues.apache.org/jira/browse/HADOOP-17643
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.7.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> WASB driver uses meta data on blobs to denote permission, whether its a place
> holder 0 sized blob for dir etc.
> For storage migration users uses Azcopy, it copies the blobs but will cause
> the metadata keys to get changed to camel case. As per discussion with MSFT
> Azcopy team, this is a known issue and technical limitation. This is what
> Azcopy team explained
> "For context, blob metadata is implemented with HTTP headers. They are case
> insensitive but case preserving.
> There is a known issue with the Go language. The HTTP client that it provides
> does this case modification to the response headers before we can read the
> raw values, so the destination metadata keys have a different casing than the
> source. We’ve reached out to the Go Team in the past but weren’t successful
> in convincing them to change the behaviour. We don’t have a short term
> solution right now"
> So propose to change the metadata key checks to do case insensitive checks.
> May be make case insensitive check configurable with defaults to false for
> compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
