I've found this in the logs: 014-09-16 11:00:31,287 INFO datanode.DataNode (SignalLogger.java:register(91)) - registered UNIX signal handlers for [TERM, HUP, INT] 2014-09-16 11:00:31,521 WARN common.Util (Util.java:stringAsURI(56)) - Path /hadoop/hdfs/data should be specified as a URI in configuration files. Please update hdfs configuration. 2014-09-16 11:00:31,523 WARN common.Util (Util.java:stringAsURI(56)) - Path should be specified as a URI in configuration files. Please update hdfs configuration. 2014-09-16 11:00:31,523 WARN common.Util (Util.java:stringAsURI(56)) - Path /data/hdfs should be specified as a URI in configuration files. Please update hdfs configuration. 2014-09-16 11:00:32,277 WARN datanode.DataNode (DataNode.java:checkStorageLocations(1941)) - Invalid dfs.datanode.data.dir /usr/lib/hadoop : EPERM: Operation not permitted at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:226) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:629) at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:467) at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:126) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:142) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:1896) at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:1938) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1920) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1812) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1859) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2035) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2059)
/hadoop/hdfs/data is the original default value. /data/hdfs is the the path I have added. All the documentation says it can be a comma-delimited list of paths but this log is complaining it's not a URI? When it's /hadoop/hdfs/data on its own it starts fine...? Regards, Charles On 16 September 2014 12:08, Charles Robertson <[email protected]> wrote: > Hi Susheel, > > Tried that - same result. DataNode still not starting. > > Thanks, > Charles > > On 16 September 2014 11:49, Susheel Kumar Gadalay <[email protected]> > wrote: > >> The VERSION file has to be same across all the data nodes directories. >> >> So I suggested to copy it as it is using OS command and start data node. >> >> On 9/16/14, Charles Robertson <[email protected]> wrote: >> > Hi Susheel, >> > >> > Thanks for the reply. I'm not entirely sure what you mean. >> > >> > When I created the new directory on the new volume I simply created an >> > empty directory. I see from the existing data node directory that it >> has a >> > sub-directory called current containing a file called VERSION. >> > >> > Your advice is to create the 'current' sub-directory and copy the >> VERSION >> > file across to it without changes? I see it has various guids, and so >> I'm >> > worried about it clashing with the VERSION file in the other data >> > directory. >> > >> > Thanks, >> > Charles >> > >> > On 16 September 2014 10:57, Susheel Kumar Gadalay <[email protected]> >> > wrote: >> > >> >> Is it something to do current/VERSION file in data node directory. >> >> >> >> Just copy from the existing directory and start. >> >> >> >> On 9/16/14, Charles Robertson <[email protected]> wrote: >> >> > Hi all, >> >> > >> >> > I am running out of space on a data node, so added a new volume to >> the >> >> > host, mounted it and made sure the permissions were set OK. Then I >> >> updated >> >> > the 'DataNode Directories' property in Ambari to include the new path >> >> > (comma separated, i.e. '/hadoop/hdfs/data,/data/hdfs'). Next I >> >> > restarted >> >> > the components with stale configs for that host, but the DataNode >> >> wouldn't >> >> > come back up, reporting 'connection refused'. When I remove the new >> >> > data >> >> > directory path from the property and restart, it starts fine. >> >> > >> >> > What am I doing wrong? >> >> > >> >> > Thanks, >> >> > Charles >> >> > >> >> >> > >> > >
