[
https://issues.apache.org/jira/browse/IGNITE-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355390#comment-15355390
]
Ivan Veselovsky commented on IGNITE-3185:
-----------------------------------------
Vladimir, code works.
My comments on the suggested solution.
1) I would not recommend to probe invalid locations (I mean probes like "Probe
for CDH with HADOOP_HOME set to parent directory."). I understand that this is
to cover mistake in documentation, but if user has an invalid env setting, he'd
better get an error, because other parts of the system may malfunction.
2) If HADOOP_HOME is set to an existing directory, but no probed combination
exists, user gets error message like "Failed to resolve Hadoop classpath
because HADOOP_HOME environment variable points to invalid directory:
/usr/hdp/current/hadoop-client". It should be made more exact, since the
directory is not "invalid".
3) If HADOOP_HOME/PREFIX is not set, absolute standard distributions (HDP, CDH,
BigTop) locations are not probed. This is a bit strange, since HADOOP_HOME is
not mandatory at all (IGNITE-3184).
> Hadoop: Improve Hadoop JAR search logic.
> ----------------------------------------
>
> Key: IGNITE-3185
> URL: https://issues.apache.org/jira/browse/IGNITE-3185
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: 1.5.0.final
> Reporter: Vladimir Ozerov
> Assignee: Ivan Veselovsky
> Fix For: 1.7
>
>
> *Problem*
> Currently our {{HadoopClassLoader}} is trying to find common/hdfs/mapred JARs
> automatically if {{HADOOP_HOME}} is set. However, it checks only for standard
> Apache Hadoop distribution paths (/share/hadoop/*). For other distributions
> (CDH, HDP), user has to bother with additional env variables manually.
> *Solution*
> Try to search for the following folders as well:
> $HADOOP_HOME/hadoop
> $HADOOP_HOME/hadoop-hdfs
> $HADOOP_HOME/hadoop-mapreduce
> If all these folders are found, then this is either HDP or CDH deployment and
> we can create the classpath without additional env vars.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
