Hi Dale,
Search engines look for robots.txt behind this url: /robots.txt
Where exactly you store this file on the file system does not matter as
much.
What matters is how you configure your routing in either Apache (common) or
in Django's urls.py (less commonly done).
In my case I use the Nginx webserver (similar to Apache). I set it up to
simply return a string when clients hit that url:
location /robots.txt {
return 200 "User-agent: *\nDisallow:\n";
}
Which in readable text would look something like:
User-agent: *
Disallow:
To disallow:
location /robots.txt {
return 200 "User-agent: *\nDisallow: /\n";
}
(note the forward slash after Disallow:)
This will probably look a little different in Apache, but I hope this will
get you started.
Best,
Vincent
On Friday, 2 November 2018 11:29:51 UTC+1, Dale Lloyd wrote:
>
> We are running Arches 4.3.3, installed by pip, on Ubuntu 16.04, served by
> Apache.
>
> If i wanted to use a robots.txt file, where would I put it?
>
> I am not sure where the root directory for the website is. I tried putting
> it in <arches project directory>/templates, next to index.htm, but that
> does not work.
>
>
--
-- To post, send email to archesproject@googlegroups.com. To unsubscribe, send
email to archesproject+unsubscr...@googlegroups.com. For more information,
visit https://groups.google.com/d/forum/archesproject?hl=en
---
You received this message because you are subscribed to the Google Groups
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to archesproject+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
