Manu Zhang created AIRFLOW-2901:
-----------------------------------
Summary: WebHdfsSensor doesn't support HDFS HA
Key: AIRFLOW-2901
URL: https://issues.apache.org/jira/browse/AIRFLOW-2901
Project: Apache Airflow
Issue Type: Improvement
Components: hooks
Reporter: Manu Zhang
If HDFS is configured with HA, we cannot use WebHdfsSensor to check for file
existence since WebHdfs cannot resolve the name service ID. Consider using
[pyarrow.hdfs|https://arrow.apache.org/docs/python/filesystems.html] as a
replacement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
