[
https://issues.apache.org/jira/browse/HADOOP-17643?focusedWorklogId=606815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-606815
]
ASF GitHub Bot logged work on HADOOP-17643:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jun/21 08:27
Start Date: 04/Jun/21 08:27
Worklog Time Spent: 10m
Work Description: surendralilhore commented on a change in pull request
#2972:
URL: https://github.com/apache/hadoop/pull/2972#discussion_r644756616
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
##########
@@ -1698,19 +1721,32 @@ private static void storeLinkAttribute(CloudBlobWrapper
blob,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
}
- private static String getLinkAttributeValue(CloudBlobWrapper blob)
+ private String getLinkAttributeValue(CloudBlobWrapper blob)
throws UnsupportedEncodingException {
- String encodedLinkTarget = getMetadataAttribute(blob,
+ String encodedLinkTarget = getMetadataAttribute(blob.getMetadata(),
LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
return decodeMetadataAttribute(encodedLinkTarget);
}
- private static boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
+ private boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
HashMap<String, String> metadata = blob.getMetadata();
- return null != metadata
- && (metadata.containsKey(IS_FOLDER_METADATA_KEY) || metadata
- .containsKey(OLD_IS_FOLDER_METADATA_KEY));
+ if (null != metadata) {
Review comment:
Can we reuse **getMetadataAttribute()** API here ?
Something like this :
return null!=getMetadataAttribute(metadata, IS_FOLDER_METADATA_KEY,
OLD_IS_FOLDER_METADATA_KEY);
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
##########
@@ -1698,19 +1721,32 @@ private static void storeLinkAttribute(CloudBlobWrapper
blob,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
}
- private static String getLinkAttributeValue(CloudBlobWrapper blob)
+ private String getLinkAttributeValue(CloudBlobWrapper blob)
throws UnsupportedEncodingException {
- String encodedLinkTarget = getMetadataAttribute(blob,
+ String encodedLinkTarget = getMetadataAttribute(blob.getMetadata(),
LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY,
OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
return decodeMetadataAttribute(encodedLinkTarget);
}
- private static boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
+ private boolean retrieveFolderAttribute(CloudBlobWrapper blob) {
HashMap<String, String> metadata = blob.getMetadata();
- return null != metadata
- && (metadata.containsKey(IS_FOLDER_METADATA_KEY) || metadata
- .containsKey(OLD_IS_FOLDER_METADATA_KEY));
+ if (null != metadata) {
Review comment:
Can we reuse **getMetadataAttribute()** API here ?
Something like this :
return null!=getMetadataAttribute(blob.getMetadata(),
IS_FOLDER_METADATA_KEY, OLD_IS_FOLDER_METADATA_KEY);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 606815)
Time Spent: 1h 10m (was: 1h)
> WASB : Make metadata checks case insensitive
> --------------------------------------------
>
> Key: HADOOP-17643
> URL: https://issues.apache.org/jira/browse/HADOOP-17643
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.7.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> WASB driver uses meta data on blobs to denote permission, whether its a place
> holder 0 sized blob for dir etc.
> For storage migration users uses Azcopy, it copies the blobs but will cause
> the metadata keys to get changed to camel case. As per discussion with MSFT
> Azcopy team, this is a known issue and technical limitation. This is what
> Azcopy team explained
> "For context, blob metadata is implemented with HTTP headers. They are case
> insensitive but case preserving.
> There is a known issue with the Go language. The HTTP client that it provides
> does this case modification to the response headers before we can read the
> raw values, so the destination metadata keys have a different casing than the
> source. We’ve reached out to the Go Team in the past but weren’t successful
> in convincing them to change the behaviour. We don’t have a short term
> solution right now"
> So propose to change the metadata key checks to do case insensitive checks.
> May be make case insensitive check configurable with defaults to false for
> compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
