[
https://issues.apache.org/jira/browse/HADOOP-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17940857#comment-17940857
]
ASF GitHub Bot commented on HADOOP-19518:
-----------------------------------------
manika137 commented on code in PR #7564:
URL: https://github.com/apache/hadoop/pull/7564#discussion_r2028078596
##########
hadoop-tools/hadoop-azure/dev-support/testrun-scripts/config_support.sh:
##########
@@ -0,0 +1,140 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+FILE=$1
+
+if [ ! -f "$FILE" ]; then
+ echo "Error: File '$FILE' not found. Exiting...."
+ exit 1
+fi
+
+if [[ "$1" != *.xml ]]; then
+ echo "The file provided is not an XML file. Exiting...."
+ exit 1
+fi
+
+OUTPUT_FILE="abfs-converted-config.xml"
+cp "$FILE" "$OUTPUT_FILE"
+
+contactTeamMsg="For any queries or support, kindly reach out to us at
'aska...@microsoft.com'."
+endpoint=".dfs."
+printf "Select 'HNS' if you're migrating to ABFS driver with Hierarchical
Namespace enabled account,
+ or 'Non-HNS' if you're migrating with Non-Hierarchical Namespace
(FNS) account. \n"
+printf "WARNING: Please ensure the correct option is chosen as it will affect
the configuration changes made to the file. \n"
+printf "If you are unsure, follow the instructions below to check from Azure
Portal: \n"
+printf "* Go to the Azure Portal and navigate to your storage account. \n"
+printf "* In the left-hand menu, select 'Overview' section and look for
'Properties'. \n"
+printf "* Under 'Blob service', check if 'Hierarchical namespace' is enabled
or disabled. \n"
+echo "$contactTeamMsg"
+select namespaceType in "HNS" "NonHNS"
+do
+ case $namespaceType in
+ HNS)
+ xmlstarlet ed -L -i '//configuration/property[1]' -t elem -n
property -v '' \
+ -s '//configuration/property[1]' -t elem -n name -v
'fs.azure.account.hns.enabled' \
+ -s '//configuration/property[1]' -t elem -n value -v 'true'
"$OUTPUT_FILE"
+ break;
+ ;;
+ NonHNS)
+ endpoint=".blob."
+ break;
+ ;;
+ *)
+ echo "Invalid selection. Please try again. Exiting..."
+ exit 1;
+ ;;
+ esac
+done
+
+# Mapping for renaming configurations
+declare -A rename_configs_map=(
+ ["autothrottling.enable"]="enable.autothrottling"
#fs.azure.autothrottling.enable to fs.azure.enable.autothrottling
+ ["rename.dir"]="rename.key" # fs.azure.atomic.rename.dir to
fs.azure.atomic.rename.key
+ ["block.blob.buffered.pread.disable"]="buffered.pread.disable"
#fs.azure.block.blob.buffered.pread.disable to fs.azure.buffered.pread.disable
+ ["fs.azure.sas"]="fs.azure.sas.fixed.token."
#fs.azure.sas.CONTAINER_NAME.ACCOUNT_NAME to
fs.azure.sas.fixed.token.CONTAINER_NAME.ACCOUNT_NAME
+ ["check.block.md5"]="enable.checksum.validation" #fs.azure.check.block.md5
to fs.azure.enable.checksum.validation
+)
+
+# Configs not supported in ABFS
+unsupported_configs_list=(
+ "fs.azure.page.blob.dir"
+ "fs.azure.block.blob.with.compaction.dir"
+ "fs.azure.store.blob.md5"
+)
+
+# Configurations not required in ABFS Driver and can be removed
+obsolete_configs_list=(
Review Comment:
The configs supported and present in both ABFS, WASB remain as it is (except
for endpoint change if required)
Script only makes changes for renames, deleting the obsoletes and error for
unsupported ones
> ABFS: [FnsOverBlob] WASB to ABFS Migration Config Support Script
> ----------------------------------------------------------------
>
> Key: HADOOP-19518
> URL: https://issues.apache.org/jira/browse/HADOOP-19518
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0, 3.4.1
> Reporter: Manika Joshi
> Assignee: Manika Joshi
> Priority: Major
> Labels: pull-request-available
>
> The legacy WASB driver has been deprecated and is no longer recommended for
> use. To support customer onboard for migration from WASB to ABFS driver,
> we've introduced a script to help with the configuration changes required for
> the same.
> The script requires the configuration file (in XML format) used for WASB and
> would generate configuration file required for ABFS driver respectively.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
