Hi Patrick,
It looks like the website is browser sniffing, and rejecting anything
with 'libwww' or 'libcurl' in the User-Agent field. Here are some
tests with curl that show the blocking of libwww/libcurl, but not
'foobar':
$ curl -A 'libwww' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 406 Not Acceptable
$ curl -A 'libcurl' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 406 Not Acceptable
$ curl -A 'foobar' -I http://www.davestrailerpage.co.uk/
HTTP/1.1 200 OK
Possibly the site is trying to deter rogue robots and scrapers, but
filtering based on the User-Agent field won't be very effective, since
it is easily changed (for example from the Lynx options page).
Cheers - Chuck
http://chuck.habilis.net
On Apr 16, 2009, at 4:06 AM, Patrick Vervoorn wrote:
Hello,
My apologies for sending this 'out of the blue', since I did not
subscribe to the lynx-dev list.
However, I'm seeing a problem which is rather Lynx-specific, since
viewing the same URL, on the same machine, but using Lynx, works
rather ok.
The full story is in the email below which I sent to Dave, but since
w3m works fine, I'm not sure it's a problem with the site...
Regards,
Patrick.
---------- Forwarded message ----------
Date: Sun, 12 Apr 2009 17:01:51 +0200 (CEST)
From: Patrick Vervoorn <[email protected]>
To: [email protected]
Subject: Viewing Trailer page with Lynx: error
Hi there,
When I try to view your site after the latest move with Lynx, I
invariably get the following error-message:
(This is when viewing both the main site at: http://www.davestrailerpage.co.uk/
but also happens when I try to view one of the sub-pages directly,
like: http://www.davestrailerpage.co.uk/about.html)
Not Acceptable
An appropriate representation of the requested resource / could
not be
found on this server.
Additionally, a 404 Not Found error was encountered while trying
to use
an ErrorDocument to handle the request.
When using something like w3m, it is displayed. Using a graphical
browser like Firefox also works fine.
Is there anything you can do from your side to be able to view your
Trailer page using Lynx?
Best regards,
Patrick.
_______________________________________________
Lynx-dev mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/lynx-dev
_______________________________________________
Lynx-dev mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/lynx-dev
