Hi ,
Thanks in advance. Please help me out in figuring cause of the follwing
error and fixing it.
Am facing the below error while scanning a hbase table with partial RowKey
filter through MapReduce program.
Error: org.apache.hadoop.horg.apache.hadoop.hdfs.server.datanode.DataNode:
DatanodeRegistration():DataXceiver java.io.InterruptedIOException:
Interruped while waiting for IO on channel
java.nio.channels.SocketChannel[closed]
Data node on the slave node is getting shutdown on this error.
My Map reduce program is running maptsks till 95% and then failing with this
error.
I have a hadoop cluster with two mechines ,
Table Size : 652 GB (223 GB in master Node and 514GB in slave node)
System Disc details:
Node space available
---------------------------------
master ---- 22 GB
slave ---- 210 GB
------------------------------- core-site.xml -----------------------
<configuration>
<property>
<name>fs.tmp.dir</name>
<value>/home/e521596/hadoop-1.1.1/full</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://172.20.193.234:9000</value>
</property>
<property>
<name>io.sort.factor</name>
<value>15</value>
<description>More streams merged at once while sorting
files.</description>
</property>
<property>
<name>io.sort.mb</name>
<value>1000</value>
<description>Higher memory-limit while sorting data.</description>
</property>
<property>
<name>io.sort.record.percent</name>
<value>0.207</value>
<description>Higher memory-limit while sorting data.</description>
</property>
<property>
<name>io.sort.spill.percent</name>
<value>1</value>
<description>Higher memory-limit while sorting data.</description>
</property>
</configuration>
------------------------------- mapred-site.xml -----------------------
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>fedora3:9001</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>6</value>
</property>
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>6</value>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>6</value>
</property>
<property>
<name>mapred.textoutputformat.separator</name>
<value>#</value>
</property>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xms1024M -Xmx2048M</value>
</property>
</configuration>
---------------------------------------- hdfs-site.xml--------------------
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/home/e521596/hadoop-1.1.1/full/dfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/e521596/hadoop-1.1.1/full/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.datanode.max.xcievers</name>
<value>5096</value>
</property>
<property>
<name>dfs.datanode.handler.count</name>
<value>200</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>0</value>
</property>
</configuration>
---------------------------------------------------------------------
--
View this message in context:
http://apache-hbase.679495.n3.nabble.com/DataXceiver-java-io-InterruptedIOException-error-on-scannning-Hbase-table-tp4059419.html
Sent from the HBase User mailing list archive at Nabble.com.
