[
https://issues.apache.org/jira/browse/HADOOP-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089572#comment-13089572
]
Alejandro Abdelnur commented on HADOOP-7563:
--------------------------------------------
Eric, the *compiled* scope in Maven actually means that the JAR will be bundled
with the application, it is a misleading name but I guess it stuck for
historical reasons.
The assemblies are doing a funny logic to exclude/include JARs in lib, the
purpose of using wildcard instead the concrete artifacts to exclude/include is
that the same assembly is used by common & hdfs (as per your feedback), and
eventually it should be leveraged by mapreduce.
Currently the assembly does:
{code}
<fileSets>
...
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>/share/hadoop/${hadoop.component}</outputDirectory>
<includes>
<include>${project.artifactId}-${project.version}.jar</include>
<include>${project.artifactId}-${project.version}-tests.jar</include>
<include>${project.artifactId}-${project.version}-sources.jar</include>
<include>${project.artifactId}-${project.version}-test-sources.jar</include>
</includes>
</fileSet>
...
</fileSets>
<dependencySets>
<dependencySet>
<outputDirectory>/share/hadoop/${hadoop.component}/lib</outputDirectory>
<unpack>false</unpack>
<scope>runtime</scope>
<useProjectArtifact>false</useProjectArtifact>
<excludes>
<exclude>org.apache.ant:*:jar</exclude>
<exclude>org.apache.hadoop:hadoop-*:jar</exclude>
<exclude>jdiff:jdiff:jar</exclude>
</excludes>
</dependencySet>
</dependencySets>
{code}
IMO, the assembly should do:
{code}
<fileSets>
...
</fileSets>
<dependencySets>
<dependencySet>
<outputDirectory>/share/hadoop/${hadoop.component}/lib</outputDirectory>
<unpack>false</unpack>
<scope>runtime</scope>
<useProjectArtifact>false</useProjectArtifact>
<excludes>
<exclude>org.apache.ant:*:jar</exclude>
<exclude>jdiff:jdiff:jar</exclude>
</excludes>
</dependencySet>
</dependencySets>
{code}
What's different?
* Hadoop JARs don't have an special handling, they end up in lib/
* Hadoop 'tests' JAR is not bundled with the distribution (*)
* Hadoop 'sources' JARs are not bundled with the distribution (*)
(*: they are all published to Apache Maven repository, and IDEs download them
from there automatically, the IDE will not look at the distribution location)
> hadoop-config.sh setup HADOOP_HDFS_HOME and HADOOP_MAPRED_HOME incorrectly
> --------------------------------------------------------------------------
>
> Key: HADOOP-7563
> URL: https://issues.apache.org/jira/browse/HADOOP-7563
> Project: Hadoop Common
> Issue Type: Bug
> Components: scripts
> Affects Versions: 0.23.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.23.0
>
> Attachments: HADOOP-7563-1.patch, HADOOP-7563.patch
>
>
> HADOOP_HDFS_HOME and HADOOP_MAPRED_HOME was set to
> HADOOP_PREFIX/share/hadoop/hdfs and HADOOP_PREFIX/share/hadoop/mapreduce.
> This setup confuses the location of hdfs and mapred scripts. Instead the
> script should look for hdfs and mapred script in HADOOP_PREFIX/bin.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
