correction: >>ls /data/d3/yarn/local ls: reading directory /data/d3/yarn/local: Input/output error
Thanks On Wed, May 6, 2015 at 7:53 PM, Lavkesh Lahngir <[email protected]> wrote: > Hello, > > If a resource is localised on a disk and that disk has gone bad after > localising, subsequent containers are not able to find the resource and NM > does not download it again. > The problem is stat system call succeeds on the bad path which causes > file.exists() to return true. > But ls on the path returns an IO error. > > LocalResourcesTrackerImpl.java > case REQUEST: > if (rsrc != null && (!isResourcePresent(rsrc))) { > LOG.info("Resource " + rsrc.getLocalPath() > + " is missing, localizing it again"); > removeResource(req); > rsrc = null; > } > if (null == rsrc) { > rsrc = new LocalizedResource(req, dispatcher); > localrsrc.put(req, rsrc); > } > break; > > isResourcePresent() calls file.exists() which calls stat64 natively which > returns true.. But the disk actually is bad, and there is no possibility of > reading/writing on that path. > > example: > >>stat /data/d3/yarn/local > > File: `/data/d3/yarn/local' > Size: 4096 Blocks: 8 IO Block: 4096 directory > Device: 830h/2096d Inode: 107307009 Links: 3 > Access: (0755/drwxr-xr-x) Uid: ( 110/ yarn) Gid: ( 118/ hadoop) > Access: 2014-11-18 13:57:19.000000000 +0000 > Modify: 2014-11-19 11:15:15.000000000 +0000 > Change: 2014-11-19 11:15:15.000000000 +0000 > Birth: - > > and ls says > > ls: reading directory /data/d3/mapred: Input/output error > > > Any thoughts? > > Thanks >
