FYI, if u disable your block cache - you will ask for "Index" blocks for every single request. So such a high rate of request is plausible for Index blocks even when your requests are totally random on your data.
Varun On Mon, Jul 8, 2013 at 4:45 PM, Viral Bajaria <[email protected]>wrote: > Good question. When I looked at the logs, it's not clear from it whether > it's reading a meta or data block. Is there any kind of log line that > indicates that ? Given that it's saying that it's ready from a startOffset > I would assume this is a data block. > > A question that comes to mind, is this read doing a seek to that position > directly or is it going to cache the block ? Looks like it is not caching > the block if it's reading directly from a given offset. Or am I wrong ? > > Following is a sample line that I used while debugging: > 2013-07-08 22:58:55,221 DEBUG org.apache.hadoop.hdfs.DFSClient: New > BlockReaderLocal for file > /mnt/data/current/subdir34/subdir26/blk_-448970697931783518 of size > 67108864 startOffset 13006577 length 54102287 short circuit checksum true > > On Mon, Jul 8, 2013 at 4:37 PM, Jean-Daniel Cryans <[email protected] > >wrote: > > > Do you know if it's a data or meta block? >
