Hi all
I found some warning in the log, and it may the bug of hdfs. Is there anyone can confirm that? I make a simple test case and the output is attached. I test it on hadoop-dist-2.0.0-alpha Best Regards -- Zhanwei Wang
2012-08-09 17:17:27,370 WARN hdfs.DFSClient
(BlockReaderLocal.java:newBlockReader(202)) - BlockReaderLocal: Removing
BP-887243813-10.32.218.87-1344402731086:blk_-1774368773209473449_6857 from
cache because local file
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449
could not be opened.
2012-08-09 17:17:27,372 WARN hdfs.DFSClient
(DFSInputStream.java:blockSeekTo(471)) - Failed to connect to
/10.32.218.87:50010 for block, add to deadNodes and continue.
java.io.FileNotFoundException:
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449_6857.meta
(No such file or directory)
java.io.FileNotFoundException:
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449_6857.meta
(No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at
org.apache.hadoop.hdfs.BlockReaderLocal.newBlockReader(BlockReaderLocal.java:179)
at
org.apache.hadoop.hdfs.DFSClient.getLocalBlockReader(DFSClient.java:701)
at
org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:787)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:444)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:577)
at java.io.DataInputStream.read(DataInputStream.java:132)
at org.apache.hadoop.hdfs.TestAppend.main(TestAppend.java:63)
2012-08-09 17:17:27,373 INFO hdfs.DFSClient
(DFSInputStream.java:chooseDataNode(651)) - Could not obtain block
BP-887243813-10.32.218.87-1344402731086:blk_-1774368773209473449_6857 from any
node: java.io.IOException: No live nodes contain current block. Will get new
block locations from namenode and retry...
2012-08-09 17:17:27,374 WARN hdfs.DFSClient
(DFSInputStream.java:chooseDataNode(666)) - DFS chooseDataNode: got # 1
IOException, will wait for 880.5585500550588 msec.
2012-08-09 17:17:28,262 WARN hdfs.DFSClient
(BlockReaderLocal.java:newBlockReader(202)) - BlockReaderLocal: Removing
BP-887243813-10.32.218.87-1344402731086:blk_-1774368773209473449_6857 from
cache because local file
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449
could not be opened.
2012-08-09 17:17:28,263 WARN hdfs.DFSClient
(DFSInputStream.java:blockSeekTo(471)) - Failed to connect to
/10.32.218.87:50010 for block, add to deadNodes and continue.
java.io.FileNotFoundException:
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449_6857.meta
(No such file or directory)
java.io.FileNotFoundException:
/Users/wangzw/workspace/p4home/wangz11-mac/cdb2/private/ardcn/hd/dist/data/data/current/BP-887243813-10.32.218.87-1344402731086/current/finalized/subdir0/blk_-1774368773209473449_6857.meta
(No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at
org.apache.hadoop.hdfs.BlockReaderLocal.newBlockReader(BlockReaderLocal.java:179)
at
org.apache.hadoop.hdfs.DFSClient.getLocalBlockReader(DFSClient.java:701)
at
org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:787)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:444)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:577)
at java.io.DataInputStream.read(DataInputStream.java:132)
at org.apache.hadoop.hdfs.TestAppend.main(TestAppend.java:63)
2012-08-09 17:17:28,264 INFO hdfs.DFSClient
(DFSInputStream.java:chooseDataNode(651)) - Could not obtain block
BP-887243813-10.32.218.87-1344402731086:blk_-1774368773209473449_6858 from any
node: java.io.IOException: No live nodes contain current block. Will get new
block locations from namenode and retry...
2012-08-09 17:17:28,264 WARN hdfs.DFSClient
(DFSInputStream.java:chooseDataNode(666)) - DFS chooseDataNode: got # 2
IOException, will wait for 6374.522215048361 msec.
2012-08-09 17:17:34,647 INFO hdfs.DFSClient
(DFSInputStream.java:blockSeekTo(448)) - Successfully connected to
/10.32.218.87:50010 for block -1774368773209473449
ok
TestReadAndAppend.java
Description: Binary data
