Hello All,
I am simply fetching data from robots.txt of a url. Below is my code.
siteurl = siteurl.rstrip("/")
--
http://mail.python.org/mailman/listinfo/python-list
Hello All,
I am simply fetching data from robots.txt of a url. Below is my code.
siteurl = siteurl.rstrip("/")
--
http://mail.python.org/mailman/listinfo/python-list