On Thu, 16 Apr 2009, Chuck Houpt wrote:
Hi Patrick,
It looks like the website is browser sniffing, and rejecting anything with
'libwww' or 'libcurl' in the User-Agent field. Here are some tests with curl
that show the blocking of libwww/libcurl, but not 'foobar':
$ curl -A 'libwww' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 406 Not Acceptable
$ curl -A 'libcurl' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 406 Not Acceptable
$ curl -A 'foobar' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 200 OK
Possibly the site is trying to deter rogue robots and scrapers, but filtering
based on the User-Agent field won't be very effective, since it is easily
changed (for example from the Lynx options page).
Aha, I did not know that. I changed the User-Agent field in my lynx.cfg to
just plain 'Lynx' and the site is displaying again.
I also forwarded your comments to Dave himself, perhaps he can do
something about it. This limitation was introduced when he recently moved
the site to a new host, so perhaps it's something which was introduced by
his new hoster.
Thanks for the explanation in any case. As to the other response: my
apologies for not searching through the archives before posting (emailing)
my question; too late for that now though...
Cheers - Chuck
http://chuck.habilis.net
Regards,
Patrick.
On Apr 16, 2009, at 4:06 AM, Patrick Vervoorn wrote:
Hello,
My apologies for sending this 'out of the blue', since I did not subscribe
to the lynx-dev list.
However, I'm seeing a problem which is rather Lynx-specific, since viewing
the same URL, on the same machine, but using Lynx, works rather ok.
The full story is in the email below which I sent to Dave, but since w3m
works fine, I'm not sure it's a problem with the site...
Regards,
Patrick.
---------- Forwarded message ----------
Date: Sun, 12 Apr 2009 17:01:51 +0200 (CEST)
From: Patrick Vervoorn <[email protected]>
To: [email protected]
Subject: Viewing Trailer page with Lynx: error
Hi there,
When I try to view your site after the latest move with Lynx, I invariably
get the following error-message:
(This is when viewing both the main site at:
http://www.davestrailerpage.co.uk/ but also happens when I try to view one
of the sub-pages directly, like:
http://www.davestrailerpage.co.uk/about.html)
Not Acceptable
An appropriate representation of the requested resource / could not be
found on this server.
Additionally, a 404 Not Found error was encountered while trying to use
an ErrorDocument to handle the request.
When using something like w3m, it is displayed. Using a graphical browser
like Firefox also works fine.
Is there anything you can do from your side to be able to view your Trailer
page using Lynx?
Best regards,
Patrick.
_______________________________________________
Lynx-dev mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/lynx-dev
_______________________________________________
Lynx-dev mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/lynx-dev
