[
https://issues.apache.org/jira/browse/HIVE-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gaoshijie updated HIVE-3743:
----------------------------
Description:
A short period of time operate load command, happen this:
the log from hadoop 'hadoop-user-datanode-host.log'
java.io.IOException: xceiverCount 256 exceeds the limit of concurrent xcievers
255
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:92)
at java.lang.Thread.run(Thread.java:619)
load command like this:
LOAD DATA INPATH '/tmp/1' INTO TABLE tb1;
LOAD DATA INPATH '/tmp/2' INTO TABLE tb1;
....
LOAD DATA INPATH '/tmp/255' INTO TABLE tb1;
LOAD DATA INPATH '/tmp/256' INTO TABLE tb1;
....
LOAD DATA INPATH '/tmp/1000' INTO TABLE tb1;
8 minutes after per operate.
the log from hadoop 'hadoop-user-datanode-host.log'
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
DatanodeRegistration(host:50010,
storageID=DS-161359523-host-50010-1353919878809, infoPort=50075,
ipcPort=50020):DataXceiver
java.net.SocketTimeoutException: 480000 millis timeout while waiting for
channel to be ready for write. ch : java.nio.channels.SocketChannel[
was:
A short period of time operate load command, happen this:
the log from hadoop 'hadoop-user-datanode-host.log'
java.io.IOException: xceiverCount 256 exceeds the limit of concurrent xcievers
255
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:92)
at java.lang.Thread.run(Thread.java:619)
load command like this:
LOAD DATA INPATH '/tmp/1' INTO TABLE tb1;
LOAD DATA INPATH '/tmp/2' INTO TABLE tb1;
....
LOAD DATA INPATH '/tmp/255' INTO TABLE tb1;
LOAD DATA INPATH '/tmp/256' INTO TABLE tb1;
....
LOAD DATA INPATH '/tmp/1000' INTO TABLE tb1;
> DML Operations , load operation error.
> --------------------------------------
>
> Key: HIVE-3743
> URL: https://issues.apache.org/jira/browse/HIVE-3743
> Project: Hive
> Issue Type: Bug
> Reporter: Gaoshijie
>
> A short period of time operate load command, happen this:
> the log from hadoop 'hadoop-user-datanode-host.log'
> java.io.IOException: xceiverCount 256 exceeds the limit of concurrent
> xcievers 255
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:92)
> at java.lang.Thread.run(Thread.java:619)
> load command like this:
> LOAD DATA INPATH '/tmp/1' INTO TABLE tb1;
> LOAD DATA INPATH '/tmp/2' INTO TABLE tb1;
> ....
> LOAD DATA INPATH '/tmp/255' INTO TABLE tb1;
> LOAD DATA INPATH '/tmp/256' INTO TABLE tb1;
> ....
> LOAD DATA INPATH '/tmp/1000' INTO TABLE tb1;
> 8 minutes after per operate.
> the log from hadoop 'hadoop-user-datanode-host.log'
> ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
> DatanodeRegistration(host:50010,
> storageID=DS-161359523-host-50010-1353919878809, infoPort=50075,
> ipcPort=50020):DataXceiver
> java.net.SocketTimeoutException: 480000 millis timeout while waiting for
> channel to be ready for write. ch : java.nio.channels.SocketChannel[
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
