Got it, Harsh, now I see it all. Thank you, Mark
On Fri, Jan 24, 2014 at 7:49 PM, Harsh J <[email protected]> wrote: > The right syntax is to use "-files -blocks -locations", so it drills > down all the way. You are not "missing" a feature - this has existed > for as long as I've known HDFS. > > In Rodrigo's output, he's seeing a BlockPool ID, which is not > equivalent to a location, but just carries an IP in it for > identification purposes. > > On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner <[email protected]> > wrote: > > Yes, Rodrigo, > > > > that's what I was looking for. So in my install I somehow don't have it > at > > all. Was asked by my students, so I got the answer. > > > > Mark > > > > > > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo > > <[email protected]> wrote: > >> > >> Mark, > >> > >> there we go ;-) > >> > >> Rodrigo Nascimento > >> Systems Engineer @ Brazil > >> Mobile +55 11 991.873.810 > >> > >> Sent from my iPhone > >> > >> Begin forwarded message: > >> > >> From: "Nascimento, Rodrigo" <[email protected]> > >> Date: 24 de janeiro de 2014 15:59:33 BRST > >> To: "[email protected]" <[email protected]> > >> Subject: RE: hdfs fsck -locations > >> > >> Hi Mark, > >> > >> It is a sample from my sandbox. Your question is about the part that is > in > >> RED at the output below, right? > >> > >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations > >> Connecting to namenode via http://sandbox.hortonworks.com:50070 > >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path > >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 > >> . > >> /user/ambari-qa/passwd: Under replicated > >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target > Replicas > >> is 3 but found 1 replica(s). > >> Status: HEALTHY > >> Total size: 1708 B > >> Total dirs: 0 > >> Total files: 1 > >> Total symlinks: 0 > >> Total blocks (validated): 1 (avg. block size 1708 B) > >> Minimally replicated blocks: 1 (100.0 %) > >> Over-replicated blocks: 0 (0.0 %) > >> Under-replicated blocks: 1 (100.0 %) > >> Mis-replicated blocks: 0 (0.0 %) > >> Default replication factor: 3 > >> Average block replication: 1.0 > >> Corrupt blocks: 0 > >> Missing replicas: 2 (66.666664 %) > >> Number of data-nodes: 1 > >> Number of racks: 1 > >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds > >> > >> > >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY > >> [root@sandbox ~]# > >> > >> Rod Nascimento > >> > >> ________________________________ > >> From: Nascimento, Rodrigo [[email protected]] > >> Sent: Friday, January 24, 2014 3:34 PM > >> To: [email protected] > >> Subject: RE: hdfs fsck -locations > >> > >> I’m not seeing locations flag yet. > >> > >> > >> > >> Rod Nascimento > >> > >> Systems Engineer @ Brazil > >> > >> > >> > >> People don’t buy WHAT you do. They buy WHY you do it. > >> > >> > >> > >> From: Mark Kerzner [mailto:[email protected]] > >> Sent: Friday, January 24, 2014 3:16 PM > >> To: Hadoop User > >> Subject: Re: hdfs fsck -locations > >> > >> > >> > >> Sorry, did not copy the full command > >> > >> > >> > >> hdfs fsck /user/mark/data/word_count.csv -locations > >> > >> Connecting to namenode via http://mark-7:50070 > >> > >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 > >> > >> .Status: HEALTHY > >> > >> Total size: 7217 B > >> > >> Total dirs: 0 > >> > >> Total files: 1 > >> > >> Total blocks (validated): 1 (avg. block size 7217 B) > >> > >> Minimally replicated blocks: 1 (100.0 %) > >> > >> Over-replicated blocks: 0 (0.0 %) > >> > >> Under-replicated blocks: 0 (0.0 %) > >> > >> Mis-replicated blocks: 0 (0.0 %) > >> > >> Default replication factor: 1 > >> > >> Average block replication: 1.0 > >> > >> Corrupt blocks: 0 > >> > >> Missing replicas: 0 (0.0 %) > >> > >> Number of data-nodes: 1 > >> > >> Number of racks: 1 > >> > >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds > >> > >> > >> > >> > >> > >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY > >> > >> > >> > >> > >> > >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J <[email protected]> wrote: > >> > >> Sorry, but what was the question? I also do not see a locations option > >> flag. > >> > >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" <[email protected]> > wrote: > >> > >> Here is an example > >> > >> > >> > >> hdfs fsck /user/mark/data/word_count.csv > >> > >> Connecting to namenode via http://mark-7:50070 > >> > >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 > >> > >> .Status: HEALTHY > >> > >> Total size: 7217 B > >> > >> Total dirs: 0 > >> > >> Total files: 1 > >> > >> Total blocks (validated): 1 (avg. block size 7217 B) > >> > >> Minimally replicated blocks: 1 (100.0 %) > >> > >> Over-replicated blocks: 0 (0.0 %) > >> > >> Under-replicated blocks: 0 (0.0 %) > >> > >> Mis-replicated blocks: 0 (0.0 %) > >> > >> Default replication factor: 1 > >> > >> Average block replication: 1.0 > >> > >> Corrupt blocks: 0 > >> > >> Missing replicas: 0 (0.0 %) > >> > >> Number of data-nodes: 1 > >> > >> Number of racks: 1 > >> > >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds > >> > >> > >> > >> > >> > >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J <[email protected]> wrote: > >> > >> Hi Mark, > >> > >> Yes, the locations are shown as IP. > >> > >> > >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner < > [email protected]> > >> wrote: > >> > Hi, > >> > > >> > hdfs fsck -locations > >> > > >> > is supposed to show every block with its location? Is location the ip > of > >> > the > >> > datanode? > >> > > >> > Thank you, > >> > Mark > >> > >> > >> -- > >> Harsh J > >> > >> > >> > >> > > > > > > > > -- > Harsh J >
