[
https://issues.apache.org/jira/browse/ACCUMULO-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791015#comment-13791015
]
Christopher Tubbs commented on ACCUMULO-884:
--------------------------------------------
Well, the local files are created as the data node, so they would have
permissions for the data node. Unless the data node process is writing files
that are world readable on the local filesystem (I hope not, but I expect this
to be controllable with umask or similar, or by controlling read or execute
permissions on the directory where the local files are stored), it should be
the case that nobody else can do local reads on those files. It was my
understanding that this is how the feature was implemented... that the client
read the local files directly.
[~kturner] and I were talking and it seems there is a possibility this could be
made to work, via native libraries where the data node uses its permissions to
get a file handle, and passes that to the client via some IPC, but to the best
of my knowledge this is not the way this feature was implemented in HDFS. (And,
I'm not sure what the SElinux implications would be for such a mechanism, if
that was turned on, either.)
Another way they could have implemented this is by sending through some local
unix socket or some other streamlined "local" RPC call to the data node, but I
don't think they did that either... and it would still go through the data node
if it did, and I'm not sure that would be any more performant than a regular
RPC call that just happened to route within the local machine.
> Take advantage of short circuit read for local files
> ----------------------------------------------------
>
> Key: ACCUMULO-884
> URL: https://issues.apache.org/jira/browse/ACCUMULO-884
> Project: Accumulo
> Issue Type: Improvement
> Components: docs
> Reporter: Billie Rinaldi
> Assignee: Keith Turner
>
> This is a new feature in hadoop 1.0.x and some versions of 0.22 and 0.23. It
> allows a client to read directly from disk instead of through a DataNode when
> the data is stored locally. Enabling it involves setting two configuration
> parameters, the first in hdfs-site.xml and the second in accumulo-site.xml.
> We should make sure this works with Accumulo and recommend it in the
> documentation.
> - dfs.block.local-path-access.user is the key in datanode configuration to
> specify the user allowed to do short circuit read.
> - dfs.client.read.shortcircuit is the key to enable short circuit read at the
> client side configuration.
> See HDFS-2246 and http://hbase.apache.org/book/perf.hdfs.configs.html for
> more information.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
