uranusjr commented on a change in pull request #17946:
URL: https://github.com/apache/airflow/pull/17946#discussion_r700023924
##########
File path: airflow/www/views.py
##########
@@ -2924,6 +2925,16 @@ def tree_data(self):
# avoid spaces to reduce payload size
return htmlsafe_json_dumps(tree_data, separators=(',', ':'))
+ @expose('/robots.txt')
+ @action_logging
+ def robots(self):
+ """
+ Returns a robots.txt file for blocking certain search engine crawlers.
This mitigates some
+ of the risk associated with exposing Airflow to the public internet,
however it does not
+ address the real security risks associated with such a deployment.
+ """
+ return send_from_directory(current_app.static_folder, 'robots.txt')
Review comment:
The physical file is easy to maintain, but not easy to find for people
not intrinsically faimiliar with Flask. And the file is a line-delimited pure
text file, so the advantage over a multiple line string
```python
"""\
User-agent: *
Disallow: /
"""
```
is marginal at best.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
