[
https://issues.apache.org/jira/browse/ARROW-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tiger068 updated ARROW-4820:
----------------------------
Description:
in hdfs.py, the method _derive_hadoop_classpath add jar files under
$HADOOP_HOME into hadoop classpath,but the hadoop config directory is not
contained in classpath.
when hadoop HA mode enabled,the hdfs uri like this: hdfs://ns
when the HADOOP_CONF_DIR directory is not in the hadoop classpath,the libhdfs
can not locate the right hdfs-site.xml, in the HA mode, hdfs service name was
parsed as host name ,it is not correct
!image-2019-03-11-16-39-25-195.png!
was:
in hdfs.py, the method _derive_hadoop_classpath add jar files under
$HADOOP_HOME into hadoop classpath,but the hadoop config directory is not
contained in classpath.
when hadoop HA mode enabled,the hdfs uri like this: hdfs://ns
when the HADOOP_CONF_DIR directory is not in the hadoop classpath,the libhdfs
can not locate the right hdfs-site.xml, in the HA mode, hdfs service name was
passed as host name ,it is not correct
!image-2019-03-11-16-39-25-195.png!
> [Python] hadoop class path derived not correct
> ----------------------------------------------
>
> Key: ARROW-4820
> URL: https://issues.apache.org/jira/browse/ARROW-4820
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.12.0, 0.12.1
> Reporter: Tiger068
> Priority: Major
> Fix For: 0.13.0
>
> Attachments: image-2019-03-11-16-39-25-195.png
>
>
> in hdfs.py, the method _derive_hadoop_classpath add jar files under
> $HADOOP_HOME into hadoop classpath,but the hadoop config directory is not
> contained in classpath.
>
> when hadoop HA mode enabled,the hdfs uri like this: hdfs://ns
> when the HADOOP_CONF_DIR directory is not in the hadoop classpath,the libhdfs
> can not locate the right hdfs-site.xml, in the HA mode, hdfs service name
> was parsed as host name ,it is not correct
> !image-2019-03-11-16-39-25-195.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
