Hi Samir, That was it - I changed ownership of the /usr/lib/hadoop dir to hdfs:hadoop and tried again and the DataNode has started successfully.
Thank you! Charles On 16 September 2014 13:47, Samir Ahmic <[email protected]> wrote: > Hi Charles, > > From log it looks like that DataNode process don't have permissions to > write to "/usr/lib/hadoop" dir. Can you check permissions on " > /usr/lib/hadoop" for user under which DataNode process is started. > (probably hdfs user but not sure). > > Cheers > Samir > > On Tue, Sep 16, 2014 at 2:40 PM, Charles Robertson < > [email protected]> wrote: > >> I've found this in the logs: >> >> 014-09-16 11:00:31,287 INFO datanode.DataNode >> (SignalLogger.java:register(91)) - registered UNIX signal handlers for >> [TERM, HUP, INT] >> 2014-09-16 11:00:31,521 WARN common.Util (Util.java:stringAsURI(56)) - >> Path /hadoop/hdfs/data should be specified as a URI in configuration files. >> Please update hdfs configuration. >> 2014-09-16 11:00:31,523 WARN common.Util (Util.java:stringAsURI(56)) - >> Path should be specified as a URI in configuration files. Please update >> hdfs configuration. >> 2014-09-16 11:00:31,523 WARN common.Util (Util.java:stringAsURI(56)) - >> Path /data/hdfs should be specified as a URI in configuration files. Please >> update hdfs configuration. >> 2014-09-16 11:00:32,277 WARN datanode.DataNode >> (DataNode.java:checkStorageLocations(1941)) - Invalid dfs.datanode.data.dir >> /usr/lib/hadoop : >> EPERM: Operation not permitted >> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) >> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:226) >> at >> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:629) >> at >> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:467) >> at >> org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:126) >> at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:142) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:1896) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:1938) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1920) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1812) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1859) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2035) >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2059) >> >> /hadoop/hdfs/data is the original default value. /data/hdfs is the the >> path I have added. All the documentation says it can be a comma-delimited >> list of paths but this log is complaining it's not a URI? When it's >> /hadoop/hdfs/data on its own it starts fine...? >> >> Regards, >> Charles >> >> On 16 September 2014 12:08, Charles Robertson < >> [email protected]> wrote: >> >>> Hi Susheel, >>> >>> Tried that - same result. DataNode still not starting. >>> >>> Thanks, >>> Charles >>> >>> On 16 September 2014 11:49, Susheel Kumar Gadalay <[email protected]> >>> wrote: >>> >>>> The VERSION file has to be same across all the data nodes directories. >>>> >>>> So I suggested to copy it as it is using OS command and start data node. >>>> >>>> On 9/16/14, Charles Robertson <[email protected]> wrote: >>>> > Hi Susheel, >>>> > >>>> > Thanks for the reply. I'm not entirely sure what you mean. >>>> > >>>> > When I created the new directory on the new volume I simply created an >>>> > empty directory. I see from the existing data node directory that it >>>> has a >>>> > sub-directory called current containing a file called VERSION. >>>> > >>>> > Your advice is to create the 'current' sub-directory and copy the >>>> VERSION >>>> > file across to it without changes? I see it has various guids, and so >>>> I'm >>>> > worried about it clashing with the VERSION file in the other data >>>> > directory. >>>> > >>>> > Thanks, >>>> > Charles >>>> > >>>> > On 16 September 2014 10:57, Susheel Kumar Gadalay < >>>> [email protected]> >>>> > wrote: >>>> > >>>> >> Is it something to do current/VERSION file in data node directory. >>>> >> >>>> >> Just copy from the existing directory and start. >>>> >> >>>> >> On 9/16/14, Charles Robertson <[email protected]> wrote: >>>> >> > Hi all, >>>> >> > >>>> >> > I am running out of space on a data node, so added a new volume to >>>> the >>>> >> > host, mounted it and made sure the permissions were set OK. Then I >>>> >> updated >>>> >> > the 'DataNode Directories' property in Ambari to include the new >>>> path >>>> >> > (comma separated, i.e. '/hadoop/hdfs/data,/data/hdfs'). Next I >>>> >> > restarted >>>> >> > the components with stale configs for that host, but the DataNode >>>> >> wouldn't >>>> >> > come back up, reporting 'connection refused'. When I remove the new >>>> >> > data >>>> >> > directory path from the property and restart, it starts fine. >>>> >> > >>>> >> > What am I doing wrong? >>>> >> > >>>> >> > Thanks, >>>> >> > Charles >>>> >> > >>>> >> >>>> > >>>> >>> >>> >> >
