The datanode data directory permission settings applies to Apache HDFS 0.22, 0.23 and 2.x, is unrelated and unnecessary for HDFS-347 but is necessary for the older SCR. If not mentioned in the book yet, they ought to go in I think as even some other distributions (other than HDFS 1.x) included the specific security settings in their past-version releases.
The other settings (the domain socket options), which remove the need to expose the data directory permissions are part of HDFS-347, which are in a branch upstream right now, awaiting integration into an upcoming HDFS release. We could document these better options once such a release is out, as its a further gain and is also secure. On Thu, Mar 7, 2013 at 2:38 PM, Anoop Sam John <[email protected]> wrote: > Hi Byran, > This change is needed with usage of any of the open > src HDFS release or is it only in CDH? Is this related with HDFS-347? > In such a case forget abt my previous mail abt adding in book :) > > -Anoop- > ________________________________________ > From: Kevin O'dell [[email protected]] > Sent: Thursday, March 07, 2013 2:13 AM > To: [email protected] > Cc: [email protected] > Subject: Re: Odd WARN in hbase 0.94.2 > > Hi Byran, > > One of engineers wanted me to pass this recommendation along: > > The issue here is mostly likely a missing configuration. > > We need to have this configuration on both the DataNode and RegionServer: > <property> > <name>dfs.domain.socket.path</name> > <value>/var/run/hadoop-hdfs/dn._PORT</value> > </property> > <property> > <name>dfs.client.read.shortcircuit</name> > <value>true</value> > </property> > > In CDH4.1 and earlier, the DN didn't need to be configured to use SCR, > so most likely he does not have dfs.client.read.shortcircuit set in > hdfs-site.xml. Adding that should fix it > > On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault > <[email protected]>wrote: > >> Yep we do have that property set to that value. The file does not seem to >> exist when I try ls'ing it myself. I'm not sure where it comes from or how >> it should be created. >> >> >> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[email protected] >> >wrote: >> >> > Bryan, >> > >> > What permissions did you set for the SCRs? The DNs need >> > >> > <property> >> > <name>dfs.datanode.data.dir.perm</name> >> > <value>755</name> >> > </property> >> > >> > >> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault >> > <[email protected]>wrote: >> > >> > > We are running hbase 0.94.2, cdh4.2 edition. We have the local read >> > > shortcut enabled. Since updating to this version I've seen a bunch of >> > WARN >> > > messages corresponding to exceptions trying to connect to what looks >> like >> > > the local DN. >> > > >> > > As far as I can tell it doesn't appear to be having much affect on the >> RS >> > > but it'd be nice to clear it up and know for sure that it doesn't >> > indicate >> > > some problem. Has anyone ever seen this and know what it is or how to >> fix >> > > it? >> > > >> > > http://pastebin.com/PA6Y9pJN >> > > >> > > Thanks! >> > > >> > >> > >> > >> > -- >> > Kevin O'Dell >> > Customer Operations Engineer, Cloudera >> > >> > > > > -- > Kevin O'Dell > Customer Operations Engineer, Cloudera -- Harsh J
