[
https://issues.apache.org/jira/browse/IGNITE-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334440#comment-15334440
]
Ivan Veselovsky edited comment on IGNITE-3185 at 6/16/16 7:02 PM:
------------------------------------------------------------------
As I understand, in case of almost all non-apache distributions the hdfs &
mapred homes are not sub-folders of HADOOP_HOME.
HADOOP_HOME by its original sense represents hadoop client base folder
(${HADOOP_HOME}/bin/hadoop executable file assumed to exist).
So, common , hdfs and mapred base folders typically have some common root , but
this root is not a value of HADOOP_HOME.
E.g. out docs v1.0 say (
http://apacheignite.gridgain.org/v1.0/docs/installing-on-hortonworks-hdp ) :
{code}
HDP=/usr/hdp/current
export HADOOP_HOME=$HDP/hadoop-client/
export HADOOP_COMMON_HOME=$HDP/hadoop-client/
export HADOOP_HDFS_HOME=$HDP/hadoop-hdfs-client/
export HADOOP_MAPRED_HOME=$HDP/hadoop-mapreduce-client/
{code}
Note that more late version of that page defines HADOOP_HOME differently (
https://apacheignite-fs.readme.io/docs/installing-on-hortonworks-hdp ): here
HADOOP_HOME is not a client home folder.
I would suggest to implement solution described in IGNITE-483 , item "2)":
specify a directory location profile for each supported distribution. The user
will just need to chose the right profile name to get correct layout.
was (Author: iveselovskiy):
As I understand, in case of almost all non-apache distributions the hdfs &
mapred homes are not sub-folders of HADOOP_HOME.
HADOOP_HOME by its original sense represents hadoop client base folder
(${HADOOP_HOME}/bin/hadoop executable file assumed to exist).
So, common , hdfs and mapred base folders typically have some common root , but
this root is not a value of HADOOP_HOME.
E.g. out docs v1.0 say (
http://apacheignite.gridgain.org/v1.0/docs/installing-on-hortonworks-hdp ) :
{code}
HDP=/usr/hdp/current
export HADOOP_HOME=$HDP/hadoop-client/
export HADOOP_COMMON_HOME=$HDP/hadoop-client/
export HADOOP_HDFS_HOME=$HDP/hadoop-hdfs-client/
export HADOOP_MAPRED_HOME=$HDP/hadoop-mapreduce-client/
{code}
Note that more late version of that page defines HADOOP_HOME differently (
https://apacheignite-fs.readme.io/docs/installing-on-hortonworks-hdp ): here
HADOOP_HOME is not a client home folder.
I would suggest another solution: specify a directory location profile for each
supported distribution. The user will just need to chose the right profile name
to get correct layout.
> Hadoop: Improve Hadoop JAR search logic.
> ----------------------------------------
>
> Key: IGNITE-3185
> URL: https://issues.apache.org/jira/browse/IGNITE-3185
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: 1.5.0.final
> Reporter: Vladimir Ozerov
> Assignee: Ivan Veselovsky
> Fix For: 1.7
>
>
> *Problem*
> Currently our {{HadoopClassLoader}} is trying to find common/hdfs/mapred JARs
> automatically if {{HADOOP_HOME}} is set. However, it checks only for standard
> Apache Hadoop distribution paths (/share/hadoop/*). For other distributions
> (CDH, HDP), user has to bother with additional env variables manually.
> *Solution*
> Try to search for the following folders as well:
> $HADOOP_HOME/hadoop
> $HADOOP_HOME/hadoop-hdfs
> $HADOOP_HOME/hadoop-mapreduce
> If all these folders are found, then this is either HDP or CDH deployment and
> we can create the classpath without additional env vars.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
