I wonder if we'd want to be a bit more nuanced. If I click on "Download Snapshot" and then am waiting for the zip to be generated, I might hit refresh, and then I get a page that says "405 Method Not Allowed". Would it be practical to have a GET request check for status only? If there were no snapshot ready or in-progress, we'd probably need a message & link to POST a new request.
That would also allow people to share URLs (e.g. in an email or webpage) directly to the code snapshot page still. --- ** [tickets:#6595] Prevent spiders from requesting tarballs** **Status:** code-review **Labels:** stability **Created:** Thu Aug 22, 2013 03:43 PM UTC by Dave Brondsema **Last Updated:** Thu Aug 22, 2013 10:00 PM UTC **Owner:** Tim Van Steenburgh The following are examples of spiders requesting tarball creation. This is unnecessary and a waste of resources. We should make it not possible. We already have `rel=nofollow` but that apparently isn't working. I think the best solution is to require the URL to be a POST. ~~~~ "GET /p/z-i/code-0/208/tarball HTTP/1.0" 200 16400 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" "GET /p/jhotdraw/svn/729/tarball HTTP/1.0" 200 17834 "-" "msnbot/0.01 (+http://search.msn.com/msnbot.htm)" "GET /p/fourpane/git4pane/ci/ec65df3a5ff2ec7be011c0722286e766c2b76d94/tarball HTTP/1.0" 200 18137 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.3; http://www.majestic12.co.uk/bot.php?+)" "GET /u/lluct/me722-cm/ci/0aa649648a00979ad6ca9e9d61df4e44eb694259/tarball?path=/external/clang HTTP/1.0" 200 17918 "-" "YisouSpider" ~~~~ --- Sent from sourceforge.net because allura-dev@incubator.apache.org is subscribed to https://sourceforge.net/p/allura/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.
