Jose Luis Pedrosa created HADOOP-16408:
------------------------------------------
Summary: `hadoop classpath` does not export azure jars, only data
lake
Key: HADOOP-16408
URL: https://issues.apache.org/jira/browse/HADOOP-16408
Project: Hadoop Common
Issue Type: Improvement
Components: fs/azure
Affects Versions: 3.2.0
Reporter: Jose Luis Pedrosa
`hadoop classpath` does not export azure jars, only azure data lake jars
This means that when running spark applications, it will not be able to use
abfs urls (among other things).
When running:
{code}
hadoop-3.2.0/bin/hadoop classpath
{code}
It will output all the jars for *hadoop-azure-datalake*, but will not output
the jars for *hadoop-azure,* which are the ones that actually contain the FS
for the new ABFS.
{noformat}
/hadoop-3.2.0/etc/hadoop:
/hadoop-3.2.0/share/hadoop/common/lib/*:
/hadoop-3.2.0/share/hadoop/common/*:
/hadoop-3.2.0/share/hadoop/tools/lib/aliyun-sdk-oss-2.8.3.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/jdom-1.1.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/hadoop-aliyun-3.2.0.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/aws-java-sdk-bundle-1.11.375.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/hadoop-aws-3.2.0.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/azure-data-lake-store-sdk-2.2.9.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/hadoop-azure-datalake-3.2.0.jar:
/hadoop-3.2.0/share/hadoop/hdfs:
/hadoop-3.2.0/share/hadoop/hdfs/lib/*:
/hadoop-3.2.0/share/hadoop/hdfs/*:
/hadoop-3.2.0/share/hadoop/tools/lib/kafka-clients-0.8.2.1.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/lz4-1.2.0.jar:
/hadoop-3.2.0/share/hadoop/tools/lib/hadoop-kafka-3.2.0.jar:
/hadoop-3.2.0/share/hadoop/mapreduce/lib/*:
/hadoop-3.2.0/share/hadoop/mapreduce/*:
/hadoop-3.2.0/share/hadoop/tools/lib/hadoop-openstack-3.2.0.jar:
/hadoop-3.2.0/share/hadoop/yarn:
/hadoop-3.2.0/share/hadoop/yarn/lib/*:
/hadoop-3.2.0/share/hadoop/yarn/*
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
