Hi Rajiv, At runtime, can the user's program provides its credentials( username, password etc) and hdfs can verify it with the ACL entries ( username , ofcourse password cannot be stored) to grant/deny it access to the file? So, the basic question is, can the user/group be provided at runtime rather using process owner username of the client process?
Also, another reason for asking for Proxy is desire to control the access to the content ( example hide SSN, bank account etc) of the file rather than file itself. A proxy sitting in between client and datanode would achieve this as we could apply filter to the content at the proxy level. Please guide me as to if this feasible in current Hadoop architecture. Will an enhancement request to build a proxy hooks for HDFS so that we can apply more policy decisions at the proxy level make sense? thanks Rahul On Mon, Feb 9, 2015 at 4:53 PM, Rajiv Chittajallu <raj...@yahoo-inc.com> wrote: > SOCKS proxies TCP connections. It wouldn't understand L7 traffic. New HDFS > ACL feature[1] would provide additional controls. If there is a need beyond > that, it would be better as a enhancement request in HDFS than building a > proxy. > > > [1] > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Access_Control_Lists > ________________________________ > From: Rahul Shrivastava <rhshr...@gmail.com> > To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu <raj...@yahoo-inc.com> > Sent: Monday, February 9, 2015 3:24 PM > Subject: Re: Proxy for HDFS > > Hi Rajiv, > > Thanks again for quick reply. > > The use case is as follows. Please let me know how can i use HDFS > RPC+SOCKS proxy to achieve below. > > 1. Client connect to HDFS proxy using HDFS driver ( not HTTPFs or > WebHDFS).Proxy that sits between client and HDFS server. > 2. Client request to access a file from HDFS cluster. > 3. Proxy recognizes that the client does not have permission to access > the file > 4. Proxy replies back to the client with no data or access denied. > > Please note that policy that drives access control of the files is much > more complex then just ACL of the file. > > thanks > rahul > > > On Mon, Feb 9, 2015 at 3:11 PM, Rajiv Chittajallu <raj...@yahoo-inc.com> > wrote: > > Rahul, > > > >If a client can use HDFS RPC why is a Proxy required? Are the clients not > allowed to reach data nodes directly? > > > >WebHDFS + Apache Traffic server or HDFS RPC + SOCK Proxy should work. > > > >If you can share a use case, it would probably help. > > > >-rajive > > > > > >----- Original Message ----- > >From: Rahul Shrivastava <rhshr...@gmail.com> > >To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu <raj...@yahoo-inc.com> > >Cc: > > > >Sent: Monday, February 9, 2015 2:58 PM > >Subject: Re: Proxy for HDFS > > > >Thanks Rajiv. > > > >I did look into HttpFs before but i wanted a build a proxy at HDFS layer. > >This is specific for clients which do not use HTTP ( i.e. HttpFs or > >webHDFS) to talk to the HDFS cluster. That includes clients which uses > HDFS > >driver to talk to HDFS cluster. > > > >thanks > >Rahul > > > > > > > > > > > >On Mon, Feb 9, 2015 at 2:53 PM, Rajiv Chittajallu < > >raj...@yahoo-inc.com.invalid> wrote: > > > >> > >> > https://github.com/apache/hadoop/tree/branch-2.6/hadoop-hdfs-project/hadoop-hdfs-httpfs > >> > >> > >> > >> > >> ----- Original Message ----- > >> From: Rahul Shrivastava <rhshr...@gmail.com> > >> To: hdfs-dev@hadoop.apache.org > >> Cc: > >> Sent: Monday, February 9, 2015 2:35 PM > >> Subject: Proxy for HDFS > >> > >> Hi, > >> > >> I have looking in HDFS code base ( 2.6.0) for the last couple of days > for > >> any possible proxy that we could utilize to create a proxy for HDFS. > >> Is there any hook within HDFS to build a proxy around that. > >> > >> thanks > >> Rahul > >> > > >
