[
https://issues.apache.org/jira/browse/HADOOP-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553662#comment-16553662
]
Steve Loughran commented on HADOOP-15544:
-----------------------------------------
no need for a cluster
Run: mvn package -Pdist -DskipTests -Dmaven.javadoc.skip=true -DskipShade
this puts things under hadoop-dist/target/hadoop-3.2.0-SNAPSHOT
* cd there
* copy into etc/hadoop under there your log4j and core-site.xml settings
* in your ~/.hadooprc file, add what you want for Hadoop modules to always
load; here's one of mine
{code}
> cat ~/.hadooprc
hadoop_add_to_classpath_tools hadoop-aws hadoop-azure hadoop-azuredatalake
{code}
then go {{bin/hadoop fs -ls abfs://something}} to see what happens
> ABFS: validate packing, transient classpath, hadoop fs CLI
> ----------------------------------------------------------
>
> Key: HADOOP-15544
> URL: https://issues.apache.org/jira/browse/HADOOP-15544
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: dependencies.txt
>
>
> Validate the packaging and dependencies of ABFS
> * hadoop-cloud-storage artifact to export everything needed
> * {{hadoop fs -ls abfs://path}} to work in ASF distributions
> * check transient CP (e.g spark)
> Spark master;s hadoop-cloud module depends on hadoop-cloud-storage if you
> build with the hadoop-3.1 profile, so it should automatically get in there.
> Just need to check that it picks it up too
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
