[issue21475] Support the Sitemap extension in robotparser

Ned Deily added the comment: Thanks for the patch, Peter, and thanks for the PR and test, Lady Red! Merged for release in 3.8.0. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue21475] Support the Sitemap extension in robotparser

Ned Deily added the comment: New changeset 5db5c0669e624767375593cc1a01f32092c91c58 by Ned Deily (Christopher Beacham) in branch 'master': bpo-21475: Support the Sitemap extension in robotparser (GH-6883) https://github.com/python/cpython/commit/5db5c0669e624767375593cc1a01f32092c91c58 -

[issue21475] Support the Sitemap extension in robotparser

Lady Red added the comment: Sorry, wrong PR number. it is 6883, and attached to this ticket -- ___ Python tracker ___ ___ Python-bug

[issue21475] Support the Sitemap extension in robotparser

Lady Red added the comment: I wrote a test for this as it seems to have been abandoned, and opened a PR. https://github.com/python/cpython/pull/6878 -- nosy: +mcsc...@gmail.com ___ Python tracker __

[issue21475] Support the Sitemap extension in robotparser

Change by Roundup Robot : -- pull_requests: +6556 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue21475] Support the Sitemap extension in robotparser

Steven Steven added the comment: Kindly add a test for this issue -- nosy: +stevensalbert ___ Python tracker ___ ___ Python-bugs-lis

[issue21475] Support the Sitemap extension in robotparser

Stéphane Wirtel added the comment: Hi @berker and @pwirtz. could you write a test for this issue? thanks -- ___ Python tracker ___ ___

[issue21475] Support the Sitemap extension in robotparser

Change by Raymond Hettinger : -- versions: +Python 3.8 -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue21475] Support the Sitemap extension in robotparser

Changes by STINNER Victor : -- dependencies: +Rewrite test_robotparser ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue21475] Support the Sitemap extension in robotparser

Berker Peksag added the comment: Peter didn't write a test because issue 25497 (test_robotparser rewrite) needs to be committed first. See msg253016 in issue 25400 for more information. -- stage: needs patch -> patch review ___ Python tracker

[issue21475] Support the Sitemap extension in robotparser

Stéphane Wirtel added the comment: Add a test with your patch. Thank you -- nosy: +matrixise ___ Python tracker ___ ___ Python-bugs-l

[issue21475] Support the Sitemap extension in robotparser

Peter Wirtz added the comment: Here is a patch that provides support for the Sitemap extension. -- keywords: +patch Added file: http://bugs.python.org/file40791/robotparser_site_maps_v1.patch ___ Python tracker ___

[issue21475] Support the Sitemap extension in robotparser

Berker Peksag added the comment: issue 25400 is not a blocker of this, so feel free to write a patch. -- ___ Python tracker ___ ___ Py

[issue21475] Support the Sitemap extension in robotparser

Peter Wirtz added the comment: I would like to tackle this issue. Should I wait for issue25400 to be resolved first? -- nosy: +pwirtz ___ Python tracker ___

[issue21475] Support the Sitemap extension in robotparser

Berker Peksag added the comment: The Crawl-delay part(issue 16099) is now committed. -- stage: -> needs patch title: Support the Sitemap and Crawl-delay extensions in robotparser -> Support the Sitemap extension in robotparser versions: +Python 3.6 -Python 3.5