Hi robots.txt fan, I just wondered what happened if I removed robots.txt from my server, to reduce the whole discussion to absurdity. :)
Best regards, ToBeFree On Thu, Jun 22, 2017, 13:08 Kristian Fiskerstrand < kristian.fiskerstr...@sumptuouscapital.com> wrote: > On 06/22/2017 10:40 AM, robots.txt fan wrote: > > Kristian, you have responded to this thread, I believe you manage the > first one on the list. Is there a reason why only /status is blocked and > not /pks? > > > > https://sks-keyservers.net (blocks /status, but not /pks) > > The real reason is that /pks didn't exist when the robots.txt file was > created, so I've [added it now], granted more for site resource > management reasons than privacy reasons. > > From a privacy perspective robots.txt doesn't make sense, the data is > already public, bad actors ignore robots.txt and crawl the site just the > same; and the full data set is available and part of regular workflow > for bootstrapping own servers. > > References: > [added it now] > > https://git.sumptuouscapital.com/?p=sks-keyservers-pool.git;a=commit;h=b98e7522990961541165dfc23781a45a1a5e05a9 > > -- > ---------------------------- > Kristian Fiskerstrand > Blog: https://blog.sumptuouscapital.com > Twitter: @krifisk > ---------------------------- > Public OpenPGP keyblock at hkp://pool.sks-keyservers.net > fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 > ---------------------------- > "Whatever you do in life, surround yourself with smart people who'll > argue with you." > John Wooden > > _______________________________________________ > Sks-devel mailing list > Sks-devel@nongnu.org > https://lists.nongnu.org/mailman/listinfo/sks-devel >
_______________________________________________ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
