Hello there,
I'm setting up a new cluster and I notice in my RS startup logs an ominous
warning.
2016-11-03 21:52:14,624 WARN [RS_LOG_REPLAY_OPS-r103u3:16020-0]
shortcircuit.DomainSocketFactory: error creating DomainSocket
java.net.ConnectException: connect(2) error: Permission denied when trying
to connect to '/var/run/hadoop-hdfs/dn.socket'
My hdfs-site.xml (not not hbase-site.xml) has:
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.client.read.shortcircuit.streams.cache.size</name>
<value>4096</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/var/run/hadoop-hdfs/dn.socket</value>
</property>
lsof on the RS process shows me that libhadoop.so has been loaded.
$ sudo lsof -p $(cat /var/run/hbase/hbase-hbase-regionserver.pid) | grep
'libhadoop\.so'
java 33581 hbase mem REG 252,0 137720 2492109
/usr/hdp/2.3.2.0-2950/hadoop/lib/native/libhadoop.so.1.0.0
Given the hbase user's group membership,
$ sudo -u hbase groups
hbase hadoop
Permission on the socket itself looks suspicious
$ sudo ls -la /var/run/hadoop-hdfs
total 0
drwx------ 2 hdfs root 60 Nov 2 22:56 .
drwxr-xr-x 23 root root 760 Nov 3 21:52 ..
srw-rw-rw- 1 hdfs hdfs 0 Nov 2 22:56 dn.socket
Our book has this word of caution: "Be careful about permissions for the
directory that hosts the shared domain socket; dfsclient will complain if
open to other than the hbase user." But this seems inaccurate in that I'm
seeing HDFS complain if the directory is open to a user other than hdfs.
So what's the correct solution here? Do I add hdfs user to the hdfs group?
That sounds too permissive.
Thanks,
Nick
