Hi,
> An alternate would be to put directives in the header if we are not
> operating at the root of the server. robots.txt is just simpler :)
hm - this is a good point: if pootle has no control over the /robots.txt file,
but still wants to prevent robots from going through all the linked data, then
we should add the appropriate meta tag to the header of each page
(http://www.robotstxt.org/wc/meta-user.html).
The attached patch prevents robots from harvesting all templates for links.
The dynamically created robots.txt would be somehow obsolete then, but it could
stay there to keep away robots which do not understand the meta tag.
What do you think?
Lars
Index: templates/projectadmin.html
===================================================================
--- templates/projectadmin.html (Revision 5939)
+++ templates/projectadmin.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle Admin: ${project.name}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/adminlanguages.html
===================================================================
--- templates/adminlanguages.html (Revision 5939)
+++ templates/adminlanguages.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title>Pootle Languages Admin Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/adminindex.html
===================================================================
--- templates/adminindex.html (Revision 5939)
+++ templates/adminindex.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title>Pootle Admin Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/projectlangadmin.html
===================================================================
--- templates/projectlangadmin.html (Revision 5939)
+++ templates/projectlangadmin.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle Admin: ${language.name} ${project.name}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/language.html
===================================================================
--- templates/language.html (Revision 5939)
+++ templates/language.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle: ${language.name}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<script type="text/javascript" src="${baseurl}js/sorttable.js"></script>
Index: templates/register.html
===================================================================
--- templates/register.html (Revision 5939)
+++ templates/register.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle Registration</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/home.html
===================================================================
--- templates/home.html (Revision 5939)
+++ templates/home.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">User Page for: ${username}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/projects.html
===================================================================
--- templates/projects.html (Revision 5939)
+++ templates/projects.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/error.html
===================================================================
--- templates/error.html (Revision 5939)
+++ templates/error.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Error</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<meta py:if="refresh" http-equiv="Refresh" content="${refresh}; URL=${refreshurl}" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
Index: templates/translatepage.html
===================================================================
--- templates/translatepage.html (Revision 5939)
+++ templates/translatepage.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle: translating ${project.name} into ${language.name}: ${pofilename}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<script src="${baseurl}js/autoexpand.js" type="text/javascript">
Index: templates/options.html
===================================================================
--- templates/options.html (Revision 5939)
+++ templates/options.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Options for: ${username}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/pootlepage.html
===================================================================
--- templates/pootlepage.html (Revision 5939)
+++ templates/pootlepage.html (Arbeitskopie)
@@ -4,6 +4,7 @@
<head>
<title>Pootle Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/activate.html
===================================================================
--- templates/activate.html (Revision 5939)
+++ templates/activate.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle Account Activation</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/redirect.html
===================================================================
--- templates/redirect.html (Revision 5939)
+++ templates/redirect.html (Arbeitskopie)
@@ -7,6 +7,7 @@
<title py:content="pagetitle">Redirecting...</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="Refresh" content="${refresh}; URL=${refreshurl}" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/adminprojects.html
===================================================================
--- templates/adminprojects.html (Revision 5939)
+++ templates/adminprojects.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title>Pootle Projects Admin Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/languages.html
===================================================================
--- templates/languages.html (Revision 5939)
+++ templates/languages.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/project.html
===================================================================
--- templates/project.html (Revision 5939)
+++ templates/project.html (Arbeitskopie)
@@ -7,6 +7,7 @@
<title py:content="pagetitle">Pootle: ${project.name}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="description" content="${meta_description}" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<script type="text/javascript" src="${baseurl}js/sorttable.js"></script>
Index: templates/index.html
===================================================================
--- templates/index.html (Revision 5939)
+++ templates/index.html (Arbeitskopie)
@@ -8,6 +8,7 @@
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="description" content="${meta_description}" />
<meta name="keywords" content="${', '.join(keywords)}" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/fileindex.html
===================================================================
--- templates/fileindex.html (Revision 5939)
+++ templates/fileindex.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Pootle: Project ${project.name}, Language ${language.name}</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<script type="text/javascript" src="${baseurl}js/sorttable.js"></script>
Index: templates/login.html
===================================================================
--- templates/login.html (Revision 5939)
+++ templates/login.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title py:content="pagetitle">Login to Pootle</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/adminusers.html
===================================================================
--- templates/adminusers.html (Revision 5939)
+++ templates/adminusers.html (Arbeitskopie)
@@ -6,6 +6,7 @@
<title>Pootle User Admin Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
Index: templates/about.html
===================================================================
--- templates/about.html (Revision 5939)
+++ templates/about.html (Arbeitskopie)
@@ -8,6 +8,7 @@
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="description" content="${meta_description}" />
<meta name="keywords" content="${', '.join(keywords)}" />
+ <meta name="robots" content="index,nofollow" />
<link rel="stylesheet" type="text/css" href="${baseurl}pootle.css" />
<link rel="shortcut icon" href="${baseurl}favicon.ico" />
<!--[if lt IE 7.]>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Translate-pootle mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/translate-pootle
