So my understanding is that this log file was opened at 7:29 and then something happened at 10:12:55 as something triggered the recovery on that block. It triggered a recovery of the block with the new name being blk_1213779416283711358_54249
It seems that that process was started by the DFS Client at 10:12:55 but the RS log starts at 10:14. Would it be possible to see what was before that? Also it would be nice to have a view for those blocks on all the datanodes. It would be nice to do this debugging on IRC is it can require a lot of back and forth. J-D On Mon, Apr 11, 2011 at 11:22 AM, Eran Kutner <[email protected]> wrote: > There wasn't an attachment, I pasted all the lines from all the NN logs that > contain that particular block number inline. > > As for CPU/IO, first there is nothing else running on those servers, second, > CPU utilization on the slaves at peak load was around 40% and disk IO > utilization less than 20%. That's the strange thing about it (I have another > thread going about the performance), there is no bottleneck I could identify > and yet the performance was relatively low, compared to the numbers I see > quoted for HBase in other places. > > The first line of the NN log says: > BLOCK* NameSystem.allocateBlock: > /hbase/.logs/hadoop1-s01.farm-ny.gigya.com,60020,1302185988579/hadoop1-s01.farm-ny.gigya.com%3A60020.1302434963279.blk_1213779416283711358_54194 > So it looks like a file name is: > /hbase/.logs/hadoop1-s01.farm-ny.gigya.com,60020,1302185988579/hadoop1-s01.farm-ny.gigya.com%3A60020.1302434963279 > > Is there a better way to associate a file with a block? > > -eran > > >
