[
https://issues.apache.org/jira/browse/HDFS-9651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956207#comment-15956207
]
Akira Ajisaka commented on HDFS-9651:
-------------------------------------
+1, I've verified the robots.txt can be accessed via
http://host:port/robots.txt. Checking this in.
> All web UIs should include a robots.txt file
> --------------------------------------------
>
> Key: HDFS-9651
> URL: https://issues.apache.org/jira/browse/HDFS-9651
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Lars Francke
> Assignee: Lars Francke
> Priority: Minor
> Attachments: HDFS-9651.1.patch, HDFS-9651.2.patch
>
>
> Similar to HDFS-330. So that public UIs don't get crawled.
> I can provide a patch that includes a simple robots.txt. Another alternative
> is probably a Filter that provides one automatically for all UIs but I don't
> have time to do that.
> If anyone wants to take over please go ahead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
