Jason Sharpee wrote:
: It would even be more handy, IMO, to also be able to specify the remote
: link depth at a specific local depth value.
:
: ie. For Linux Today:
:
: Local-Depth=2
: Local-Depth-Level=1 Remote-Depth=0
: Local-Depth-Level=2 Remote-Depth=1
:
: Would grab the local contents page without banner's offsite links, and
: when a story link is traversed to depth 2, it would then allow spidering
: out to the complete story link.
:
: Would this be usefull or could this be handled another existing way?
I had a similar problem here (think downloading comments from
slashdot, but only those with score >=4. Or recoding from utf-8
to iso 8859-2. Or sending cokies or Accept-Language: or Accept-Charset:
HTTP headers).
I think the best solution would be to use wget or some other generic
web spider to mirror this, and then include the file:// URLs
in your home.html.
-Yenya
--
| Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> |
| GPG: ID 1024/D3498839 Fingerprint 0D99A7FB206605D7 8B35FCDE05B18A5E |
| http://www.fi.muni.cz/~kas/ Czech Linux Homepage: http://www.linux.cz/ |
|\ Pruning the incoming mailbox after being 10 days off-line. Sorry /|
| \ for the delayed reply. / |
