Thanks for your response Chris.
>> I'm curious how this is a new problem after migration from HttpFs to
WebHDFS
With HttpFs, gateway admin users are allowed to proxy some limited set of
users.
They are not allowed to mock a root/dfsadmin user and do some fatal stuff
("rmr /").
With WebHDFS, thats not the case. I can send a request as any user
including root.
>> I would think this kind of thing could be achieved by writing a custom
>> authentication
filter
Our changes to achieve this is also based on filter but that will get
activated based on configuration.
Just for reference, I added our filter code.
>> This is not a requirement I've heard from anyone else.
>> I'm generally reluctant to add features without a widespread need.
Agree. I wanted to listen to community before proceeding with jira/patch.
I thought this may be helpful for people running WebHDFS without security.
In worst case, I just need to maintain a inhouse patch till we enable
kerberos.
public class WebHdfsReadOnlyFilter implements ResourceFilter {
public static final Log LOG =
LogFactory.getLog(WebHdfsReadOnlyFilter.class);
public static final String DFS_WEBHDFS_READ_ONLY = "dfs.webhdfs.readonly";
private boolean readonly;
public WebHdfsReadOnlyFilter() {
HdfsConfiguration conf = new HdfsConfiguration();
readonly = conf.getBoolean(DFS_WEBHDFS_READ_ONLY, true);
if(readonly) {
LOG.info(DFS_WEBHDFS_READ_ONLY + " is set true. webhdfs is
readonly.");
} else {
LOG.warn(DFS_WEBHDFS_READ_ONLY + " is set false. write
operations are enabled over webhdfs");
}
}
@Override
public ContainerRequestFilter getRequestFilter() {
return readonly ? READONLY_FILTER : null;
}
@Override
public ContainerResponseFilter getResponseFilter() {
return null;
}
private static final ContainerRequestFilter READONLY_FILTER = new
ContainerRequestFilter() {
@Override
public ContainerRequest filter(final ContainerRequest request) {
if (!"GET".equals(request.getMethod())) {
Response.ResponseBuilder builder = null;
String response = "WebHDFS write operations are disabled.";
// Though 405 (Method Not Allowed) looks more
appropriate, we could not use it due to unavailability in Jersey apis.
// Please check apidocs for javax.ws.rs.core.Response.Status
// Also HTTP spec says, we need to set allowed method in HEADER
builder =
Response.status(Response.Status.FORBIDDEN).entity(response);
throw new WebApplicationException(builder.build());
}
return request;
}
};
@VisibleForTesting
protected void setReadonly(boolean readonly) {
this.readonly = readonly;
}
}
On 6 November 2015 at 23:01, Chris Nauroth <cnaur...@hortonworks.com> wrote:
> Hello Laxman,
>
> I'm curious how this is a new problem after migration from HttpFs to
> WebHDFS. With the HttpFs deployment architecture, were you somehow
> proxying only the read-only operations?
>
> I would think this kind of thing could be achieved by writing a custom
> authentication filter, deploying that to the HDFS classpath, and then
> pointing to it by setting dfs.web.authentication.filter in hdfs-site.xml
> to the full name of that custom authentication filter class. The logic of
> the custom authentication filter would check for only read-only operations
> and reject the others. This is a solution that wouldn't require changes
> in WebHDFS itself.
>
> This is not a requirement I've heard from anyone else. I'm generally
> reluctant to add features without a widespread need. Still, if you want
> to file an HDFS JIRA for further discussion of your proposal, there is no
> harm in that. It might end up as a "Won't Fix", or perhaps others in the
> community see it differently from me, and we'd want to proceed.
>
> Thanks for sharing the work you've done!
>
> --Chris Nauroth
>
>
>
>
> On 11/6/15, 3:02 AM, "Laxman Ch" <laxman....@gmail.com> wrote:
>
> >Hi,
> >
> >We run a cluster with security set to simple.
> >Also, to some users, we had provided http access to HDFS via HttpFS
> >gateways.
> >However, this is not scaling and we are suffering from HttpFs gateway
> >choking problem. So, we wanted to enable WebHDFS directly on hadoop. But
> >this brings in the problem of security. Any user can simply delete
> >anything. And, we can't enable immediately enable kerberos security in
> >production.
> >
> >How about introducing a configuration to make WebHDFS readonly?
> >We patched this in our clusters cleanly and its working.
> >
> >Please revert with your comments if its a good idea to push this to
> >hadoop.
> >If yes, I will create a jira and submit patch.
> >--
> >Thanks,
> >Laxman
>
>
--
Thanks,
Laxman
