Re: How to crawl every urls on a website?

[Jin Yang]

How to generate the urls list of a website? Should we put 1 on 1 into 
them? Like this?

www.apache.org/1.html
www.apache.org/2.html
www.apache.org/3.html
Have any tool or command can do this?