When playing for browser, I get:
[msql@chaos Zips]$ telnet www.arsdigita.com 80
Trying...
Connected to arsdigita.com.
Escape character is '^]'.
GET /bboard/ HTTP/1.0
Connection closed by foreign host.
[msql@chaos Zips]$
(nothing returned by server).
But when using a real browser, it works. Seems like you use cookies (yak!).
And I think that 3.1.5 connot cope with cookies, but correct me if I'm wrong.
On Wed, 7 Mar 2001, Ian Lipsky wrote:
> http://www.arsdigita.com/bboard/q-and-a?topic%5fid=21&topic=web%2fdb
>
> whenever i attempt to crawl that url, it fails.
>
> actually, if i try crawling http://www.arsdigita.com/bboard/ it fails. Its
> like it doesnt see the page at all. However, if i crawl
> http://www.arsdigita.com/, it works.
>
> Can anyone give me some suggestions as to why this might be failing? Theres
> another site that uses the same BB system and i crawl that one fine.
>
>
> _______________________________________________
> htdig-general mailing list <[EMAIL PROTECTED]>
> To unsubscribe, send a message to <[EMAIL PROTECTED]> with
>a subject of unsubscribe
> FAQ: http://htdig.sourceforge.net/FAQ.html
>
--jesse
--------------------------------------------------------------------
J. op den Brouw Johanna Westerdijkplein 75
Haagse Hogeschool 2521 EN DEN HAAG
Faculty of Engeneering Netherlands
Electrical Engeneering +31 70 4458936
-------------------- [EMAIL PROTECTED] --------------------
Linux - because reboots are for hardware changes
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
