Interesting answer thanks.
I suppose that test will share what header is being used when I remove the
option.
I find it is wonderful doing as much when I reach one of those 403
forbidden pages, works almost every time.
I am no technician, so was unclear how to simulate the googlebot element?
Thanks again,
Karen
On Thu, 3 Nov 2016, Tim Chase wrote:
On 2016-11-03 19:31, Karen Lewellen wrote:
I am wondering what the default user agent header is, if you ask
lynx not to send a user agent header?
If you uncheck the "Send user agent header", it's simply omitted from
the list of headers. You can test it at
http://headers.cloxy.net/request.php
which just echoes back the headers your browser sends. With my stock
lynx on Debian, it includes
* User-Agent: Lynx/2.8.9dev.1 libwww-FM/2.14 SSL-MM/1.4.1
GNUTLS/3.3.8
But when I go into options and turn off "Send user agent header",
that header list-item simply isn't present.
Lynx also offers you the ability to send the header but spoof its
content to pretend to be some other browser. Few sites check this any
more, but I've had the occasional site that gives truncated content
for normal browsers but gives the full content if you pretend to be
Googlebot (Google's web crawler).
-tim
_______________________________________________
Lynx-dev mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/lynx-dev
_______________________________________________
Lynx-dev mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/lynx-dev
