Thank you for Tim Armstrong, the version of impala I am using is based on 3.2, it contains the fixed IMPALA-7738. When the error "hdfsOpenFile() for <filename> at backend <hostname:port> failed to finish before the <hdfs_operation_timeout_sec> second timeout " appears, it may also appears “ Failed to submit hdfsOpenFile() for <filename> at backend <hostname:port> to thread pool after waiting <hdfs_operation_timeout_sec> seconds"
> 2020年11月3日 上午2:59，Tim Armstrong <tarmstr...@cloudera.com> 写道： > > https://issues.apache.org/jira/browse/IMPALA-7738 can mitigate this to a > large degree (it will cancel the query and log the error if the timeout is > exceeded). It is not a 100% solution since the thread pool is fixed in size > and can in theory get exhausted. > > On Sun, Nov 1, 2020 at 9:18 PM hexianqing <hexianqing...@126.com> wrote: > >> Hi all, >> I hit the issue that queries Stuck on Failed HDFS Calls and not Timing out >> several times when the Namenode is heavily loaded。 >> In Impala Known Issues, it is described as follows: >> "In Impala 3.2 and higher, if the following error appears multiple times >> in a short duration while running a query, it would mean that the >> connection between the impalad and the HDFS NameNode is in a bad state and >> hence the impalad would have to be restarted: >> "hdfsOpenFile() for <filename> at backend <hostname:port> failed to finish >> before the <hdfs_operation_timeout_sec> second timeout " >> In Impala 3.1 and lower, the same issue would cause Impala to wait for a >> long time or hang without showing the above error message. >> Apache Issue: HADOOP-15720( >> https://issues.apache.org/jira/browse/HADOOP-15720) >> Affected Versions: All versions of Impala >> Workaround: Restart the impalad in the bad state." >> I wonder if there is a way to avoid this or is there a plan to fix it, >> thank you! >>
