[issue41803] Robots.txt

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> rejected
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41803] Robots.txt

[Eric V. Smith]

Change by Eric V. Smith :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41803] Robots.txt

[Eric V. Smith]

Eric V. Smith  added the comment:

It's not clear what bug you're describing, or how it relates to Python. Can you 
provide more information, and show what behavior your seeing, and what behavior 
you're expecting?

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41803] Robots.txt

[Skyreka Agence Web]

New submission from Skyreka Agence Web :

I can't  documentation about it, but all of the robots.txt checkers I find 
behave like this. You can test on this site: http://www.skyreka.com/robots.txt, 
I believe that this is how it's implemented now in most parsers ?

--
messages: 377055
nosy: admin2
priority: normal
severity: normal
status: open
title: Robots.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com