[GitHub] [doris] icedrugs89 commented on issue #10696: [Bug] BrokerLoad导入任务出现type:ETL_RUN_FAIL; msg:errCode = 2, detailMessage = Broker list path exception. path=hdfs:xxx

Fri, 08 Jul 2022 02:18:54 -0700 


icedrugs89 commented on issue #10696:
URL: https://github.com/apache/doris/issues/10696#issuecomment-1178755340

   1、指定具体的年月日分区是可以导入的，怀疑是表的分区目录过大，进行如下验证：
       1)broker机器安装hdfs客户端环境，使用通配符*访问hive对应的所有分区文件，耗时大约在56秒
       2)查看对应的thrift中的socket代码，执行获取File的status状态可能由于hive分区数太大耗时太久超时
       
3)查看thrift-0.9.3/lib/java/src/org/apache/thrift/transport/TSocket.java中的SocketTimeout默认读写超时时间太短小于50秒
   
![image](https://user-images.githubusercontent.com/100941547/177959022-6b10291d-4846-4b94-80b9-44c534da1b8a.png)
       
4)由社区的技术团队修改FE的参数fe/fe-core/src/main/java/org/apache/doris/common/ClientPool.java增加static
 int brokerTimeoutMs = 10000;
       5)生成测试版本， 重启FE后再次验证，问题修复


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to