>From lacall <[email protected]>: > All I've got for you is something like this: > wget -r -l 2 --wait=5 --page-requisites --timestamping --convert-links > www.openbsd.org/faq
Oh Thank you very much for that but... > But the site admins might hate that Well this is a real concern for me too, the real reason I want to say I don't think it would work is because there is a possiblity of getting blacklisted when doing this, right? I would not want that. Have you already tested running this and getting the entire /faq/ website? I also do self host my own website and stuff and the reason why I do not want to do this is because I would hate people doing that to my site as well. Your concern is very much appreciated in this day and age. As much I want to run that command to solve this problem. As a site admin myself (of my own server) I WILL hate people doing that to my site, after seeing the log of one person crawling my website. and I would blacklist them from accessing my website. I think the same thing would happen to me, and I do not want to get blacklisted at all. Best for the site admins to just publish a archive of the /faq/ folder for us to download instead.
