Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
from Jason Harris: > Some rationale for why gzip encoding is enabled: > https://davidwalsh.name/check-gzip > A look at the headers and how wget behaves: > wget --no-check-certificate -S --header="Accept-Encoding: gzip" > "https://svnweb.freebsd.org/ports/head/x11/xcb-proto/files/patch-xcbgen_align.py?revision=425597=co; > What to add to your /usr/local/etc/lynx.cfg: > PREFERRED_ENCODING:NONE > HTH. Yes, that worked, files in https://svnweb.freebsd.org/ports/head/x11/xcb-proto/files downloaded true, file sizes were correct, no gzip. My thanks. But this really needs to be better documented. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
> > URL where I was stung was > https://svnweb.freebsd.org/ports/head/x11/xcb-proto/files > with two patch files in that directory. > Tested it with links 2.13. It works ok. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
> I have never seen lynx compress an uncompressed file. However, if lynx sends > a header that it can _accept_ gzip encoding, which I believe it might, the > webserver can easily gzip the contents to save bandwidth. lynx could possibly be saving that compressed content to disk, with a .gz extension... > Personally, I use elinks (and used to be its FreeBSD maintainer) way more > often than lynx. > Can you send an URL to recreate the problem? URL where I was stung was https://svnweb.freebsd.org/ports/head/x11/xcb-proto/files with two patch files in that directory. I was thinking about using links or elinks instead of lynx, but textproc/docbook-tools uses www/lynx as a dependency. If I don't want links' crude graphics implementation, elinks might be smaller and good enough. Building links with directfb option can take a long time; one is better off with Firefox or Seamonkey. I also emailed the upstream maintainer, Thomas Dickey (dic...@invisible-island.net). Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
> Thomas Mueller skrev: > > Documentation on Lynx and w3m are awful hard to find! > > I couldn't find anything on auto_uncompress or anything else that might be > > put in ~/.w3m/config. > > If the file on the server is already compressed, for instance a tarball, > > then I want to download it that way. > > But a browser/downloader has no proper business compressing a file to be > > downloaded. > For w3m you can set accept_encoding. Default is > accept_encoding gzip, compress, bzip, bzip2, deflate > Try to remove gz, bzip, bzip2 or set it to "". > If you think this is a bug you should report it upstream: > https://github.com/tats/w3m > Herbert I looked through w3m documentation and found nothing on accept_encoding. Their website suggests that w3m is more of a pager than a web browser. Elvis (enhanced vi clone) is/was also a rudimentary text-mode web browser. I don't install w3m explicitly, only if it is pulled in by the ports system as a dependency. Maybe I will take the same approach regarding lynx? I intend to check a NetBSD installation with lynx to see if that misbehaves the same way regarding gratuitous, unwanted gzip compression of downloads. I haven't used links in some time; remember a build on FreeBSD took six hours, but that was with DirectFB on a computer with 256 MB RAM. I think I need to email dic...@invisible-island.net . Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
Thomas Mueller skrev: > > Documentation on Lynx and w3m are awful hard to find! > > I couldn't find anything on auto_uncompress or anything else that might be > put in ~/.w3m/config. > > If the file on the server is already compressed, for instance a tarball, then > I want to download it that way. > > But a browser/downloader has no proper business compressing a file to be > downloaded. For w3m you can set accept_encoding. Default is accept_encoding gzip, compress, bzip, bzip2, deflate Try to remove gz, bzip, bzip2 or set it to "". If you think this is a bug you should report it upstream: https://github.com/tats/w3m -- Herbert ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
El día lunes, abril 24, 2017 a las 11:48:32p. m. +, Thomas Mueller escribió: > Curl is tricky to get right from the command line, I tried. > > I have no recent experience with links. Should I build with graphics, > without graphics, or build links1? > > Lynx and w3m people hide the documentation, but I looked through sample > lynx.cfg. > > Do I need to set > GZIP_PATH=/dev/null ? What about: lynx -dump http://www.unixarea.de/OtroLugar.txt > x.txt matthias -- Matthias Apitz, ✉ g...@unixarea.de, ⌂ http://www.unixarea.de/ ☎ +49-176-38902045 Aus "Nie wieder Krieg!" wurde "Nie wieder Krieg ohne Deutschlands Truppen" The "No wars anymore!" changed now to "No wars anymore without German battle groups!" El "¡Nunca jamás guerra!" ha cambiado a "¡Nunca jamás guerra sin tropas alemanas!" ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
Yes, I'm with Anatoly. Some browsers accept encrypted or compressed content and then try to display it. I've tried lynx (GPL2),wget (GPL3) but curl (MIT) with minimum options does the trick nicely. I use curl in place of fetch, so in a make.conf there's FETCH_CMD= /usr/local/bin/curl --create-dirs --connect-timeout 10 -m 240 --retry 1 -C - --ipv4 -O -o - curl is harder to master than wget ;) Hopefully this helps :) ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
On Mon, 24 Apr 2017 23:48:32 + "Thomas Mueller"wrote: > I have no recent experience with links. Should I build with > graphics, without graphics, or build links1? If you wish text console only, no graphics needed. Under x11 it's nice fast simple graphical browser when started as 'links -g' otherwise. If you wish graphics without x11, may experiment with direct fb and svgalib. Links1 is an old version (contemporary is 2.x). I think w3m doesn't gzip anything itself, but When talking to server it says that gzip supported. Then it receives compressed stream from server and fails (or just lazy) to decompress it. -w3m must not say to server "Accept-Encoding: gzip, deflate" if it can't gunzip -server must not send gzipped stream if above not said. And unfortunately, I think, some servers may not respect this. -w3m must gunzip it otherwise. I never used w3m, have not it istalled yet, but quick look at manual on the web gives a idea to try w3m -header "accept-encoding: identity" to explicitly say to server send everything in plain. Can you display content you trying to save on w3m screen online? If yes, may try to fight for gzipping off, but if no, you must investigate why gunzipping is not working within w3m (is it compiled against zlib or something at all?). Btw, what is the url you trying to save? > Lynx was nice before it got intimately mixed with gzip. But every browser nowadays can do "content gunzipping" due to existing web standard ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
from Anatoly: > If you wish console browser, try links > if you wish http get tool, try curl Curl is tricky to get right from the command line, I tried. I have no recent experience with links. Should I build with graphics, without graphics, or build links1? Lynx and w3m people hide the documentation, but I looked through sample lynx.cfg. Do I need to set GZIP_PATH=/dev/null ? Lynx was nice before it got intimately mixed with gzip. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
If you wish console browser, try links if you wish http get tool, try curl On Mon, 24 Apr 2017 04:54:27 + "Thomas Mueller"wrote: > > If you're just trying to grab a file, fetch(1) may prove adequate. > > (It's in base.) > > > Peace, > > david > >- > > David H. Wolfskill > > I tried fetch, but got something entirely different, the stuff on the > web page, but not the desired file. > > File compression, such as PKZIP, Infozip, gzip, bzip2, 7-zip, RAR, xz > are useful in places but can be overbearing in other places. > > Compressing man pages is more pain in the ass than benefit. > > Tom > > ___ > freebsd-ports@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to > "freebsd-ports-unsubscr...@freebsd.org" ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
> > Thomas Mueller skrev: > > I don't use lynx (text-mode web browser) much, but have run into a problem > > that I never had before. > > Lynx, and also w3m, download what are supposed to be text files and then I > > see the gzip'ed version on the hard drive. > > Lynx used to download files as is! > > I looked through "man lynx", also /usr/local/etc/lynx.cfg, and couldn't > > figure how to disable the annoying, gratuitous gzip. > > Mozilla Seamonkey and Firefox can download straight without altering. > > My previous experience was that Lynx was trustworthy and would download > > files as is. > For w3m you can try to set 'auto_uncompress 1' in ~/.w3m/config. > Herbert Documentation on Lynx and w3m are awful hard to find! I couldn't find anything on auto_uncompress or anything else that might be put in ~/.w3m/config. If the file on the server is already compressed, for instance a tarball, then I want to download it that way. But a browser/downloader has no proper business compressing a file to be downloaded. I might try on a NetBSD installation where I believe I installed lynx to compare the download automatic gzip behavior. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Downloading with lynx or w3m, how to download as is, without gratuitous gzip
Thomas Mueller skrev: > > I don't use lynx (text-mode web browser) much, but have run into a problem > that I never had before. > Lynx, and also w3m, download what are supposed to be text files and then I > see the gzip'ed version on the hard drive. > > Lynx used to download files as is! > > I looked through "man lynx", also /usr/local/etc/lynx.cfg, and couldn't > figure how to disable the annoying, gratuitous gzip. > > Mozilla Seamonkey and Firefox can download straight without altering. > > My previous experience was that Lynx was trustworthy and would download files > as is. For w3m you can try to set 'auto_uncompress 1' in ~/.w3m/config. -- Herbert ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Downloading with lynx or w3m, how to download as is, without gratuitous gzip
> If you're just trying to grab a file, fetch(1) may prove adequate. > (It's in base.) > Peace, > david >- > David H. Wolfskill I tried fetch, but got something entirely different, the stuff on the web page, but not the desired file. File compression, such as PKZIP, Infozip, gzip, bzip2, 7-zip, RAR, xz are useful in places but can be overbearing in other places. Compressing man pages is more pain in the ass than benefit. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Downloading with lynx or w3m, how to download as is, without gratuitous gzip
I don't use lynx (text-mode web browser) much, but have run into a problem that I never had before. Lynx, and also w3m, download what are supposed to be text files and then I see the gzip'ed version on the hard drive. Lynx used to download files as is! I looked through "man lynx", also /usr/local/etc/lynx.cfg, and couldn't figure how to disable the annoying, gratuitous gzip. Mozilla Seamonkey and Firefox can download straight without altering. My previous experience was that Lynx was trustworthy and would download files as is. Otherwise, what other text-mode web browsers are there in case my graphic interface (X) is not installed or is not operational? In this case, the problem arose due to a conflict reported by svn in /usr/ports/x11/xcb-proto/files https://svnweb.freebsd.org/ports/head/x11/xcb-proto/files I solved that problem with "svn revert -R" but still would like to know how to make lynx properly functional again. Lynx seems much easier to use than w3m, if I could find a way around its fetish with gzip; w3m also showed this fetish. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"