[Bug-wget] [Bug #23934] Support for subjectAltNames

2009-03-31 Thread Petr Pisar
Hello,

I have hit problem that wget doesn't support subject alternative names in HTTPS
communication. It's bug #23934 https://savannah.gnu.org/bugs/?23934.

Some time ago I wrote a patch solving this particular problem. It's attached
to the bug report
https://savannah.gnu.org/file/wget-1.11.4-subjectAltNames.diff?file_id=17403.

Could maintainer (Micah) look on it? Is it acceptable?

-- Petr


pgp1JRxumrDwi.pgp
Description: PGP signature


[Bug-wget] Re: Fwd: wget file-size parsing bug when doing FTP mirroring?

2009-04-01 Thread Petr Pisar
On 2009-04-01, Niek Bergboer niekbergb...@gmail.com wrote:

 [...snip...]
 -rw-rw-r-- � 1 guest � �everyone 91892436 Sep 13 �2003 Sent Items.dbx
 [...snip...]

 Mostly as expected. However, wget then tells me:

 The sizes do not match (local 91892436) -- retrieving.
 ftp://username:*passwo...@server/some/remote/dir/Sent%20Items.dbx

 ... and gets the entire ~90MB file again, despite the fact that the
 sizes are in fact equal. This seems to occur for all files =
 10,000,000 bytes.

Can you reproduce it?

I tried to reproduce it, but without any success. I mirrored
subdirectory with file (having same size, date and name as yours) from
vsftpd FTP server running on Linux using wget-1.11.4 to Linux client. On
second mirror I got message:

Remote file no newer than local file `router/pub/TEMP/testdir/Sent
Items.dbx' -- not retrieving.

So for some reason your wget does not check timestamp.

-- Petr





Re: [Bug-wget] download page-requisites with spanning hosts

2009-04-30 Thread Petr Pisar
On Wed, Apr 29, 2009 at 06:50:11PM -0500, Jake b wrote:
 
 The wGet command I am using:
 wget.exe -p -k -w 15
 http://forums.sijun.com/viewtopic.php?t=29807postdays=0postorder=ascstart=27330;
 
 It has 2 problems:
 
 1) Rename file:
 
 Instead of creating something like: 912.html or index.html it instead
 becomes: viewtopic@t=29807postdays=0postorder=ascstart=27330

That't normal because the server doean't provide any usefull alternative name
via HTTP headers which can be obtained using wget's option
--content-disposition.

If you want get number of the page of the gallery, you need to parse the HTML
code by hand to obtain it (e.g. using grep).

However I guess better naming conventions is the value of start URL
parameter (in your example the number 27330).

 2) images that span hosts are failing.
 
 I have page-resuisites on, but, since some pages are on tinypic, or
 imageshack, etc it is not downloading them. Meaning it looks like
 this:
 
 sijun/page912.php
   imageshack.com/1.png
   tinypic.com/2.png
   randomguyshost.com/3.png
 
 
 Because of this, I cannot simply list all domains to span. I don't
 know all the domains, since people have personal servers.
 
 How do I make wget download all images on the page? I don't want to
 recurse other hosts, or even sijun, just download this page, and all
 images needed to display it.
 
That's not easy task. Especially because all big desktop images are stored on
other servers. I think wget is not enough powerfull to do it all on its own.

I propose using other tools to extract the image ULRs and then to download them
using wget. E.g.:

wget -O - 
'http://forums.sijun.com/viewtopic.php?t=29807postdays=0postorder=ascstart=27+330'
 | grep -o -E 'http:\/\/[^]*\.(jpg|jpeg|png)' | wget -i -

This command downloads the HTML code, uses grep to find out all image files
stored on other servers (deciding throug file name extensions and absolute
addresses), and finally, it downloads such images.

There is little problem: not all of the images still exist and some servers
return dummy page instead of proper error code. So you can get non-image files
sometimes.

 [ This one is a lower priority, but someone might already know how to
 solve this ]
 3) After this is done, I want to loop to download multiple pages. It
 would be cool If I downloaded pages 900 to 912, and each pages next
 link work correctly to link to the local versions.
 
[…]
 Either way, I have a simple script that can convert 900 to 912 into
 the correct URLs, and pausing in between each request.
 
Wrap your script inside counted for-loop:

for N in $(seq 900 912); do
# variable N contains here the right number
echo $N
done

Acctually, I suppose you use some unix enviroment, where you have available
powerfull collection of external tools (grep, seq) and amazing shell scripting
abilities (like colons and loops).

-- Petr


pgptCJIEFQP9c.pgp
Description: PGP signature


Re: [Bug-wget] download page-requisites with spanning hosts

2009-04-30 Thread Petr Pisar
On Thu, Apr 30, 2009 at 03:31:21AM -0500, Jake b wrote:
 On Thu, Apr 30, 2009 at 3:14 AM, Petr Pisar petr.pi...@atlas.cz wrote:
 
  On Wed, Apr 29, 2009 at 06:50:11PM -0500, Jake b wrote:
 but i'm not sure how to tell wget that the output html file should be named.
 
wget -O OUTPUT_FILE_NAME

   How do I make wget download all images on the page? I don't want to
   recurse other hosts, or even sijun, just download this page, and all
   images needed to display it.
  
  That's not easy task. Especially because all big desktop images are stored
  on other servers. I think wget is not enough powerfull to do it all on its
  own.
 
 Are you saying because some services show a thumbnail, then click to do the
 full image? 
[…]
 Would it be simpler to say something like: download page 912, recursion
 level=1 ( or 2? ), except for non-image links. ( so it only allows recursion
 on images, ie: downloading randomguyshost.com/3.png
 
You can limit downloads according file name extentions (option -A), however
this will remove the sole main HTML file and prevent you in recursion. And
no, there is no option to download only files pointed from special HTML
element like IMG.

Without the -A option, you get a lot of useless files (reagardless spanning).

If you look on locations of files you are interested in, you will see that all
the files are located outside the Sijun domain. On every page is only small
amount of such files. Thus it's more efficient and friendly to the servers to
extract these URLs only at first and then download them only.

 But the problem that it does not span any hosts? Is there a way I can
 achieve this, if I do the same, except, allow span everybody, recurse
 lvl=1, and only recurse non-images.

There is option -H for spanning. Following wget-only command does what you
want, but as I said it produce a lot of useless requests and files.

wget -p -l 1 -H 
'http://forums.sijun.com/viewtopic.php?t=29807postdays=0postorder=ascstart=27+330'
 


  I propose using other tools to extract the image ULRs and then to download
  them using wget. E.g.:
 
 I guess I could use wget to get the html, and parse that for image tags
 manually, but, then I don't get the forum thread comments. Which isn't
 required, but would be nice.

You can do both: extract image URLs and extract comments.

 
  wget -O
  - 
  'http://forums.sijun.com/viewtopic.php?t=29807postdays=0postorder=ascstart=27+330'
  | grep -o -E 'http:\/\/[^]*\.(jpg|jpeg|png)' | wget -i -
 
 Ok, will have to try it out. ( In windows ATM so I can't pipe. )
 
AFAIK Windows shells command.com and cmd.exe supports pipes.

 Using python, and I have dual boot if needed.
 
Or you can execute programs connected through pipes in Python.

-- Petr


pgpkvyaPL0mO2.pgp
Description: PGP signature


Re: [Bug-wget] can wget uncompress gzip ?

2009-05-19 Thread Petr Pisar
On Tue, May 19, 2009 at 10:52:53AM +0200, fabrice régnier wrote:

 My web server send gzip pages.

 Wget can download this gzip pages with this command:

 # wget -S --header='Accept-Encoding: gzip,deflate' http://my_url

 But it stays gzip compressed. When i open these pages with firefox, i'd 
 like to see them in a more clear format ;)

This is interesting problem. I have web server behing slow link, thus I have
all bigger files compressed (on file system) and I send them to clients
compressed with propper Content-Encoding header.

 My questions are:
 * should wget uncompress gzip on the fly while downloading ?
I think it would be convenient for the user. (This feature should be driven on
Content-Encoding header only, not on Content-Type.)

 * should firefox should uncompress gzip while opening a local gzip file ?
Problem is, Firefox doesn't investigate the content of the file and it's
MIME type guess is based on file extension only. Firefox also doesn't
consider content encoding at all on local files. (It's pain to browse localy
saved web pages with compressed SVG images by Firefox.)

 * should i manually uncompress all gzip files before i can open them with 
 firefox ?
That's equivalent to first question: Is Content-Encoding just a transport
encoding (like base64 of quoted-printable in e-mail) or is it a permanent
mark `This file is compressed'?

Additionally, I have another approach: The MIME type and Content-Encoding can
be saved into extended file attributes and then any application can reuse this
info to figure out the type of data easily without file name extionsion or
file content magic. (This feature has been discussed on this list already and
my patch has been dismissed.)

-- Petr


pgpO5TedS7Ci4.pgp
Description: PGP signature


Re: [Bug-wget] Bug with multiname certificates

2009-05-26 Thread Petr Pisar
On Tue, May 26, 2009 at 09:51:08AM +0200, Marc MAURICE wrote:

 Trying to wget https://www.me.etsglobal.org/ returns a certificate error 
 (domain mismatch).
 However, Firefox and IE7 report no problem.
 I think wget doesn't handle certificates with several names correctly.


This is known bug #23934
http://savannah.gnu.org/bugs/?func=detailitemitem_id=23934. You are free to
test patch that is available on the bug page.

-- Petr


pgplZ67z6ganU.pgp
Description: PGP signature


Re: [Bug-wget] Wget and IPv6 link local addresses

2009-06-01 Thread Petr Pisar
On Sat, May 30, 2009 at 12:18:02PM +0100, Fabian Hugelshofer wrote:
 
 Wget 1.11.4 does not support IPv6 link local addresses. Addresses with a
 zone identifier are being rejected with Invalid IPv6 numeric address.
 
 e.g.:
 $ wget http://[fe80::1%25eth0]/
 http://[fe80::1%25eth0]/: Invalid IPv6 numeric address.
 
 %25 is encoded for '%', the delimiter for the zone identifier.
 
 Such an URL gets rejected by is_valid_ipv6_address() in host.c.

Fix is not easy because the zone identifier should be considered only during
making local sockets. It should be avoided in upper layer (HTTP, SSL, FTP)
processing.

You can try following patch which is quick and dirty hack only for HTTP.
Propper patch would require significant changes over all the code.

-- Petr

commit cd898b30ae67881df842f35e844c99afd7e9585b
Author: Petr Písař petr.pi...@atlas.cz
Date:   Mon Jun 1 20:58:38 2009 +0200

Naive IPv6 zone identifier implementation for HTTP.

diff --git a/src/host.c b/src/host.c
index fb5a2cb..48756d5 100644
--- a/src/host.c
+++ b/src/host.c
@@ -478,7 +478,12 @@ is_valid_ipv6_address (const char *str, const char *end)
 
   if (str == end)
 return false;
-  
+ 
+  /* RFC 4007: link scope address with zone identifier */
+  const char *realend = strchr(str, '%');
+  if (realend  end)
+  realend = end;
+
   /* Leading :: requires some special handling. */
   if (*str == ':')
 {
@@ -491,7 +496,7 @@ is_valid_ipv6_address (const char *str, const char *end)
   saw_xdigit = false;
   val = 0;
 
-  while (str  end)
+  while (str  realend)
 {
   int ch = *str++;
 
@@ -517,7 +522,7 @@ is_valid_ipv6_address (const char *str, const char *end)
   colonp = str + tp;
   continue;
 }
-  else if (str == end)
+  else if (str == realend)
 return false;
   if (tp  ns_in6addrsz - ns_int16sz)
 return false;
@@ -529,13 +534,13 @@ is_valid_ipv6_address (const char *str, const char *end)
 
   /* if ch is a dot ... */
   if (ch == '.'  (tp = ns_in6addrsz - ns_inaddrsz)
-   is_valid_ipv4_address (curtok, end) == 1)
+   is_valid_ipv4_address (curtok, realend) == 1)
 {
   tp += ns_inaddrsz;
   saw_xdigit = false;
   break;
 }
-
+
   return false;
 }
 
@@ -556,6 +561,16 @@ is_valid_ipv6_address (const char *str, const char *end)
   if (tp != ns_in6addrsz)
 return false;
 
+  /* Zone identifier has to be only decimal natural number in nummerical
+   * address. Othewise, it's plarform specific zone string (e.g. interface
+   * number) a and it has to be resolved. */
+  const char *ch;
+  for (ch = realend + 1; ch  end; ch++)
+{
+  if (*ch  '0' || *ch  '9')
+  return false;
+}
+
   return true;
 }
 
diff --git a/src/http.c b/src/http.c
index bdfe100..e6ee993 100644
--- a/src/http.c
+++ b/src/http.c
@@ -1508,6 +1508,13 @@ gethttp (struct url *u, struct http_stat *hs, int *dt, 
struct url *proxy)
becomes ambiguous and needs to be rewritten as Host:
[3ffe:8100:200:2::2]:1234.  */
   {
+/* Hide IPv6 zone identifier. Currently it has only meaning on link local
+ * connections and the idetifier is local node specific.
+ * XXX: This ugly hack mangles data inside struct url *u. Not thread-safe! 
*/
+char *zone_delimiter = strchr(u-host, '%');
+if (zone_delimiter)
+  *zone_delimiter = '\0';
+
 /* Formats arranged for hfmt[add_port][add_squares].  */
 static const char *hfmt[][2] = {
   { %s, [%s] }, { %s:%d, [%s]:%d }
@@ -1517,6 +1524,9 @@ gethttp (struct url *u, struct http_stat *hs, int *dt, 
struct url *proxy)
 request_set_header (req, Host,
 aprintf (hfmt[add_port][add_squares], u-host, 
u-port),
 rel_value);
+/* Unhide zone delimiter */
+if (zone_delimiter)
+  *zone_delimiter = '%';
   }
 
   if (!inhibit_keep_alive)
diff --git a/src/url.c b/src/url.c
index 5f61e35..de5e43e 100644
--- a/src/url.c
+++ b/src/url.c
@@ -717,7 +717,14 @@ url_parse (const char *url, int *error)
 
 #ifdef ENABLE_IPV6
   /* Check if the IPv6 address is valid. */
-  if (!is_valid_ipv6_address(host_b, host_e))
+  /* XXX: The host_b is not decoded yet. This is not perfect check for
+   * zone delimiter '%'. */
+  p = strchr(host_b, '%');
+  if (p  host_e)
+{
+  p = NULL;
+}
+  if (!is_valid_ipv6_address(host_b, p ? p : host_e))
 {
   error_code = PE_INVALID_IPV6_ADDRESS;
   goto error;


pgp8kXIs5NzJb.pgp
Description: PGP signature


[Bug-wget] Some flaws in wget-1.12-pre1 messages

2009-07-05 Thread Petr Pisar
Hello,

translating wget-1.12-pre1, I found some flaws in msgid's:

# TODO: msgid bug: explicit quotation
#: src/gnutls.c:293
#, c-format
msgid The certificate's owner does not match hostname '%s'\n
msgstr Jméno vlastníka certifikátu se neshoduje se jménem počítače „%s“\n

# TODO: msgid bug: missing space after colon
#: src/http.c:2785
#, c-format
msgid %s URL:%s %2d %s\n
msgstr %s URL:%s %2d %s\n

# TODO: msgid bug: old copyright year
#. TRANSLATORS: When available, an actual copyright character
#. (cirle-c) should be used in preference to (C).
#: src/main.c:836
msgid Copyright (C) 2008 Free Software Foundation, Inc.\n
msgstr Copyright © 2008 Free Software Foundation, Inc.\n

# TODO: msgid bug: explicit quotation
#: src/netrc.c:421
#, c-format
msgid %s: %s:%d: unknown token \%s\\n
msgstr %s: %s:%d: neznámý token „%s“\n


Also some text in wget --version is not internationalized (marked with ^):

$ ./src/wget --version
GNU Wget 1.12-b2525 built on linux-gnu.

+digest +ipv6 +nls +ntlm +opie +md5/openssl +https -gnutls +openssl 
+gettext +iri 

Wgetrc:
/home/petr/.wgetrc (user)
   ^^
/usr/local/etc/wgetrc (system)
  

-- Petr


pgpKaKkK5875N.pgp
Description: PGP signature


[Bug-wget] Wrong msgids in 1.12-pre3

2009-07-27 Thread Petr Pisar
Hello,

I hit some inconsistences in wget-1.12-pre3 message template:

 msgid 
 msgstr 
 Project-Id-Version: PACKAGE VERSION\n
 Report-Msgid-Bugs-To: w...@sunsite.dk\n
[…]

Here should be new mailing list address. This is msgstr however it's
distributed from po/wget.pot. 

 #: src/main.c:574
 msgid 
--auth-no-challenge Send Basic HTTP authentication information\n
without first waiting for the server's\n
challenge.\n

First letter should be lower-cased.

 #: src/main.c:434
 msgid 
   -B,  --base=URLresolves HTML input-file links (-i -F)\n
  relative to URL,\n

Last symbol should be full stop, not a comma.

-- Petr


pgp7RDEStSvDa.pgp
Description: PGP signature


Re: [Bug-wget] new alpha release 1.12-2416

2010-08-08 Thread Petr Pisar
On Sun, Aug 08, 2010 at 02:13:56PM +0200, Giuseppe Scrivano wrote:
 I have just uploaded a new alpha version of wget.
 
Great. Could you please upload new gettext template into Translation Project?
Last one is 1.12_pre6 from 2009-09-08
(http://translationproject.org/domain/wget.html). It would help translators to
get time to update translations and to get them into new stable wget release.

-- Petr


pgpQ1BkixFay6.pgp
Description: PGP signature


Re: [Bug-wget] Re: Re: Re: Storing the URL in extended attributes

2010-10-19 Thread Petr Pisar
On Tue, Oct 19, 2010 at 10:21:39PM +0200, Michelle Konzack wrote:
 Note:   Programs which disallow to disable such things are crap.
 
 So for wget it should only  be  used  if  the  user  set  explicite  a
 commandline option or in the wgetrc.  Of course, it should not activated
 hardcoded or as a default in the wgetrc.
 
Dear angry user, it's matter of configuration. You can compile curl/wget
without a feature, you can disable it in system wide configuration file,
a user can do the same in his own configuration file or explicitly in
arguments of the utility.

We can discuss what's ideal default option, you can talk about it with your
operating system vendor or you can do it yourself (yes, Gentoo rules :).

-- Petr


pgpmfkoy6TeEI.pgp
Description: PGP signature


Re: [Bug-wget] Need to handle wild card certificates

2010-12-02 Thread Petr Pisar
On Thu, Dec 02, 2010 at 10:21:29PM +0100, Giuseppe Scrivano wrote:
 I am not sure yet about the next release, I can't apply a patch because
 the author hasn't assigned copyright to the FSF yet.  I don't think
 there will be a release before 2-3 weeks.
 
This concrete server certificate provides DNS subject alternative names (SAN)
which are implemented in wget by a patch already commited into trunk. The SAN
patch has been written by me and the copyright asssignement to FSF has been
handled more than a year ago. (Otherwise Mica did not commit the code).

(BTW referencing the commit by ordinal number in bazaar VCS is nonsense as the
ordinal numbers are not global stable. The correct identifier is revision-id
`petr.pi...@atlas.cz-20091024230644-bawcvao7wi71y1ky'.)

I communicated the copyright assignement with Mica by direct e-mails. Please
ask him or FSF.)

-- Petr


pgpSHSHTCdoh9.pgp
Description: PGP signature


Re: [Bug-wget] Need to handle wild card certificates

2010-12-03 Thread Petr Pisar
On Fri, Dec 03, 2010 at 08:17:37AM +0100, Petr Pisar wrote:
 
 I communicated the copyright assignement with Mica by direct e-mails. Please
 ask him or FSF.)

Or I can send you a poor scan of the assignment signed by both parties.

-- Petr



pgpnDHMnZn8eh.pgp
Description: PGP signature


Re: [Bug-wget] Storing the URL in extended attributes

2011-09-04 Thread Petr Pisar
On Mon, Oct 18, 2010 at 12:48:14PM +0200, Petr Pisar wrote:
 On Mon, Oct 18, 2010 at 12:27:24PM +0200, Michelle Konzack wrote:
  Am 2010-10-17 22:38:51, hacktest Du folgendes herunter:
   I created a patch to store the URL inside the user xattrs of the
   downloaded file; this way, its origin can be identified afterwards.
   
   I uploaded the change to my Github account and attached the diff, and
   I am still working on portability issues, but I'd like to hear some
   opinions on this:
   
   http://github.com/wertarbyte/wget/tree/xattrurl
  
  I am right that this works only on Windos and not GNU/Linux?
  
 See my patch for wget
 http://article.gmane.org/gmane.comp.web.wget.general/7894. It works on
 GNU/Linux, IRIX and Darwin. Other option is to use libattr as the API is not
 standardized (it support IRIX and GNU/Linux only). FreeBSD has yet another
 API.
 

Just if somebody is still interested (maybe new wget maintainer), rebased
patch against 1.13.3 is still available on http://xpisar.wz.cz/wget-xattr/.

-- Petr



pgpNWXhdvliTt.pgp
Description: PGP signature


Re: [Bug-wget] Wget issues with delicious (SAN)

2012-01-31 Thread Petr Pisar
On Tue, Jan 31, 2012 at 06:08:43PM +0100, Sven Herzberg wrote:
 Hi,
 
 I used to execute a script to backup my bookmarks from delicious.com.
 However, since quite some time, wget doesn't connect to the remote site
 anymore, the error is:
 
  $ wget 'https://api.del.icio.us'
  --2012-01-31 12:58:52--  https://api.del.icio.us/
  Resolving api.del.icio.us... 184.72.40.0, 184.72.44.135, 50.18.156.75, ...
  Connecting to api.del.icio.us|184.72.40.0|:443... connected.
  ERROR: certificate common name `d-static.com' doesn't match requested host 
  name `api.del.icio.us'.
  To connect to api.del.icio.us insecurely, use `--no-check-certificate'.
  Unable to establish SSL connection.
 
 This looks like a trivial error, one might think. However, while the common
 name of the certificate indeed doesn't api.del.icio.us, it includes that
 name (among others) in the list of subject alternative names:
 
[...]
 Firefox, Chrome and Safari seem to be happy with this setup, wget isn't. Is
 there a reason behind this or is it just awaiting a patch to become fixed?
 
Subject alternative names are supported by wget since 1.13
(http://bzr.savannah.gnu.org/lh/wget/trunk/revision/2317).

-- Petr


pgpqBaP3W6jUl.pgp
Description: PGP signature


Re: [Bug-wget] alpha release 1.13.4.59-2b1dd

2012-05-27 Thread Petr Pisar
On Sat, May 26, 2012 at 03:00:26PM +0200, Giuseppe Scrivano wrote:
 I have just uploaded a new alpha version with the few changes of the
 last days:
 
 ftp://alpha.gnu.org/gnu/wget/wget-1.13.4.59-2b1dd.tar.bz2
 
 If nothing horrible happens, then I will make an official release in few
 days.
 
When are you going to publish new translation catalog template on the
Translation Project (TP)? I checked Czech translation in the tar ball and
there is 24 messages untranslated. If you want to get all translations ready
for new release, please push the wget.pot into the TP and get translators few
days for updating the translations.

-- Petr


pgptliG8EDiun.pgp
Description: PGP signature


[Bug-wget] Message faults in wget 1.13-pre1

2012-06-24 Thread Petr Pisar
Hello,

while updating Czech translation for wget 1.13-pre1, I found following
mistakes. 


# FIXME: Double dot
#: src/http.c:3121
msgid Cannot write to WARC file..\n

# FIXME: missing space after comma
#: src/main.c:1197
#, c-format
msgid 
Both --no-clobber and --convert-links were specified,only --convert-links 
will be used.\n

-- Petr


pgpSkWZNah6gx.pgp
Description: PGP signature


Re: [Bug-wget] [FEATURE-REQUEST] Pinning SSL certificates / check SSL fingerprints

2012-07-08 Thread Petr Pisar
On Sat, Jul 07, 2012 at 01:25:49PM -0600, Daniel Kahn Gillmor wrote:
 On 07/07/2012 12:50 PM, Ángel González wrote:
  On 06/07/12 01:01, pro...@secure-mail.biz wrote:
  Because SSL CA's have failed many times (Comodo, DigiNotar, ...) I wish to 
  have an option to pin a SSL certificate. The fingerprint may be optionally 
  provided through a new option.
  Have you tried using --ca-certificate option?
 
 I believe the OP wants to pin the certificate of the remote server (that
 is, the end entity certificate), whereas --ca-certificate pins the
 certificate of the issuing authority.
 
Indeed? I thought the --ca-certificate just makes the certificate trustful, so
declaring server certificate using --ca-certificate could be enought.

Though there can be problem with HTTP redirects and of course some picky TLS
libraries can insist on CA=true X.509 attribute. Also some TLS implementations
checks the server hostname against certificate name. 

So if the TLS library cannot be cheated with --ca-certificate option,
overriding root of trust in other way is good idea.

I'm just little worried about the digest algorithm. One can claim the MD5 is
too weak. There have been real successfull attacks exploiting MD5 collisions
of signed object in X.509 certificate. The ability to specify different
alogrithm is necessary.

Also remember the HTTP redirect scenario where you need to verify two (or
more) servers. It's necessary to be able to supply more pinning options.

Maybe option pinning certificate to hostname would be the best choice. No
hashes. Just supply the peer certificate content like with --ca-certificate.
E.g. `--peer-certificate example.com:/tmp/example.cert'.

-- Petr


pgpsNRlttrc9I.pgp
Description: PGP signature


Re: [Bug-wget] SINGLE QUOTE in wget output is replaced by UTF-8 value of SINGLE QUOTE

2012-08-09 Thread Petr Pisar
On Thu, Aug 09, 2012 at 04:50:46PM +0530, Avinash wrote:
 2) Have built a rpm on top of this source for FEDORA CORE 3

Do you know this is really ancient piece of software?

 3) It get installed correctly on FC3 and wget downloads files as well
 4) But the output of wget shows UTF-8 value of SINGLE QUOTE instead of
 showing SINGLE QUOTE i.e. '
 
[...]
 Can somebody please help as to why it is replacing SINGLE QUOTE by its
 UTF-8 equivalent ?

What's your locale value (run command `locale')? If it is UTF-8, then you get
UTF-8 characters. Try running `LC_ALL=C wget'.

-- Petr


pgpx2pUtohCc0.pgp
Description: PGP signature


Re: [Bug-wget] glibc detected

2014-06-28 Thread Petr Pisar
On Sat, Jun 28, 2014 at 10:39:09AM +0200, Martin Jašek wrote:
 
 wget -q -O spoje.html
 www.idos.cz/olomouc/odjezdy/\?f=Trnkova\t=Hlavní%20nádraží\submit=true
 http://www.idos.cz/olomouc/odjezdy/%5C?f=Trnkova%5Ct=Hlavn%C3%AD%20n%C3%A1dra%C5%BE%C3%AD%5Csubmit=true

Could you send the URL once again and this time as a plain text (no HTML).
Because text which ended up in the mailing list consists of two differents
addresses and both of them return 404 Not Found.

 in some cases (about one half of tries) it throws glibc detected:
 
 *** glibc detected *** wget: double free or corruption (!prev): 0x0843cb80
 ***
 === Backtrace: =
 /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xb7457f01]
 /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x72768)[0xb7459768]
 /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb745c81d]
 wget[0x8076c44]
 /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb73fde46]
 wget[0x804c931]
 (...)

Please install package with debugging data for wget and try again. And provide
longer back-trace.

Also running the command under gdb or valgrind would be benefical.

 
 I am using wget and system as following:
 wget --version
 GNU Wget 1.13.4 sestaven na systému linux-gnu.
 
There is 1.14 in the wild. Does it crash too?

-- Petr


pgpzQ9UUP8kCa.pgp
Description: PGP signature


Re: [Bug-wget] glibc detected

2014-06-28 Thread Petr Pisar
On Sat, Jun 28, 2014 at 01:43:15PM -0400, Ray Satiro wrote:
 In Ubuntu 14.04 / wget 1.15 I don't see anything unusual. Logs are 
 attached with version information and --debug output and a valgrind on 
 Ubuntu 13.10/wget 1.14 but I don't have the debugging package.. did you 
 mean wget debug package Petr?

The wget114out4.txt with valgrind output is promissing. The first reported
issue can be wget or glibc fault. The other ones look like wget fault.
However without the debugging symbols available, one cannot see which source
code line the mistake is on (valgrind shows only ??? in that case). Yes, you
need to install the wget debug package.

-- Petr



pgpapAKF2vq4p.pgp
Description: PGP signature


Re: [Bug-wget] glibc detected

2014-06-28 Thread Petr Pisar
On Sat, Jun 28, 2014 at 08:46:11PM -0400, Ray Satiro wrote:
 
 Where is the debug package? It's not in the default repos.
 
I don't know much about Ubuntu. Try to follow
https://wiki.ubuntu.com/DebuggingProgramCrash#Debug_Symbol_Packages,
otherwise ask Canonical for more help.

-- Petr


pgpBdKJGUWKKy.pgp
Description: PGP signature


Re: [Bug-wget] [PATCH] Allow to redefine ciphers list for OpenSSL

2014-07-09 Thread Petr Pisar
On Tue, Jul 08, 2014 at 10:00:24AM -0400, Tomas Hozza wrote:
 I'm afraid this is not suitable for us. We need to be able to define the
 policy somewhere in /etc, where the user is not able to change it (only
 the system administrator).

I hope can also prevent the user from running his own wget executable, or
ld-preloading modified OpenSSL library, or intercepting open(2) calls to
provide fake /etc file.

 Also the main intention to have a single place to set the policy for all
 system components, therefore wgetrc is not the right place for us.
 
What about to change wget to call OPENSSL_config(NULL) instead of setting some
hard-coded preference string. Then you can teach OpenSSL to load your /etc
configuration instead of patching each application.

-- Petr


pgpieMgSxd4PH.pgp
Description: PGP signature


Re: [Bug-wget] SSL Poodle attack

2014-10-15 Thread Petr Pisar
On Wed, Oct 15, 2014 at 11:57:47AM +0200, Tim Rühsen wrote:
 (means, the libraries defaults are used, whatever that is).
 
 Should we break compatibility and map 'auto' to TLSv1 ?
 For the security of the users.

Please no. Instead of changing each TLS program, one should patch only the TLS
library. This is the reason why why have shared libraries.

So just report the issue to your vendor, he will fix few TSL implementations
he delivers and all application will get fixed automatically.

-- Petr



pgpt4V8rcL4IP.pgp
Description: PGP signature


Re: [Bug-wget] certificate revocation lists (CRLs) #43501

2014-11-05 Thread Petr Pisar
On Wed, Nov 05, 2014 at 01:54:20PM +0100, Noël Köthe wrote:
 Am Mittwoch, den 05.11.2014, 12:48 +0100 schrieb Tim Ruehsen:
  BTW, does Debian meanwhile has a CRL infrastructure (something like 
  /etc/ssl/certs/) or is planning something like it ?
 
 I'm not aware of an infrastructure but asked the people who might know
 this (CC: to this list).
 
  Also, OCSP certificate status checking might be interesting for Wget.
 
 :) ACK.
 
Checking certificate validity is damn difficult (partial CRLs, CRL expiration,
CRL or OSCP server unavailability, caching).

There is dirmngr daemon that can do most of the things (it did not support
partial CRLs last time I checked). Unfortunatelly it has its own configuration
because it's from GnuPG project.

-- Petr


pgpxWpongBhF3.pgp
Description: PGP signature


Re: [Bug-wget] [PATCH] UTF-8-ify contributors' names in wget.texi (#40472)

2015-03-16 Thread Petr Pisar
On Mon, Mar 16, 2015 at 07:55:47PM +0100, Giuseppe Scrivano wrote:
 ./texi2pod.pl -D VERSION=1.16.3.6-b74a-dirty ./wget.texi wget.pod
 /usr/bin/pod2man --center=GNU Wget --release=GNU Wget 1.16.3.6-b74a-dirty 
 wget.pod  wget.1
 Wide character in printf at /usr/share/perl5/vendor_perl/Pod/Simple.pm line 
 541.
 wget.pod around line 2395: Non-ASCII character seen before =encoding in 
 'Nikšić'. Assuming UTF-8
 POD document had syntax errors at /usr/bin/pod2man line 69.
 
A year or so, podlators require to specify POD encoding if the POD is not
ASCII or ISO-8859-1. Future versions will default to CP1251. I recommend to
write:

=encoding utf8

above first POD line.

Also for some time, podlators have pretty verbose diagnosis and complains on
things which older versions did not complain. The actuall output depends on
how old your perl or CPAN installation is.

-- Petr


signature.asc
Description: PGP signature


Re: [Bug-wget] [bug #43799] wget should implement OCSP + OCSP stapling

2015-08-19 Thread Petr Pisar
On Wed, Aug 19, 2015 at 03:37:06PM +, Tim Ruehsen wrote:
 Regarding MITM and other attacks... did you notice that OCSP responder URLs
 are HTTP (plain text) will all the insecurity ? I never saw a HTTPS URL, did
 you ?
 
There is no need for HTTPS. The OCSP response is signed by the CA's OCSP
responder. So the problem of OCSP response integrity reduces to verifying the
OCSP response signature. Of course to verify the signature, one needs to
verify OCSP responder's certificate. But this is the same story as with CRLs.

-- Petr


signature.asc
Description: PGP signature


Re: [Bug-wget] Windows cert store support

2015-12-11 Thread Petr Pisar
On Fri, Dec 11, 2015 at 01:22:48PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 10 Dec 2015 01:12:37 +0100
> > From: Ángel González 
> > Cc: bug-wget 
> > 
> > On 09/12/15 03:06, Random Coder wrote:
> > > I'm not sure if the wget maintainers would be interested, but I've
> > > been carrying this patch around in my private builds of wget for a
> > > while.  It allows wget to load SSL certs from the default Windows cert
> > > store.
> > >
> > > The patch itself is fairly straightforward, but as it changes the
> > > default SSL behavior, and no care was taken to follow coding convents
> > > when I wrote it, so it's probably not ready for inclusion in the
> > > codebase.  Still, if it's useful, feel free to use it for ideas.
> > Wow, supporting the OS store would certainly be very cool.
> > 
> > I would probably move it to windows.c and attempt to make it also work 
> > in gnutls, but in general it looks good.
> 
> Wget compiled with GnuTLS already supports this feature: it calls
> gnutls_certificate_set_x509_system_trust when the GnuTLS library
> supports that.  gnutls_certificate_set_x509_system_trust does
> internally what the proposed patch does.
> 
> So I think this code should indeed go only to openssl.c, as gnutls.c
> already has its equivalent.
> 
AFAIK OpenSSL source contains crypto engine that delegates all operations
to Windows native cryptographical subsystem. It's only matter of default
configuration.

-- Petr


signature.asc
Description: PGP signature


Re: [Bug-wget] strange behaviour

2018-05-20 Thread Petr Pisar
On Thu, May 10, 2018 at 10:27:35AM +, VINEETHSIVARAMAN wrote:
> My server is behind a firewall and a  proxy, but when i give  2 "wget" in
> command  gives me a DNS resolution but not with the single wget !
> 
[...]
> [~]$ nslookup google.com
> 
> Non-authoritative answer:
> Name:   google.com
> Address: 74.125.24.102
> Name:   google.com
> Address: 74.125.24.101
> Name:   google.com
> Address: 74.125.24.139
> Name:   google.com
> Address: 74.125.24.113
> Name:   google.com
> Address: 74.125.24.138
> Name:   google.com
> Address: 74.125.24.100
> 
> [~]$ wget google.com --no-proxy -d
> DEBUG output created by Wget 1.14 on linux-gnu.
> 
> URI encoding = ‘UTF-8’
> Converted file name 'index.html' (UTF-8) -> 'index.html' (UTF-8)
> Converted file name 'index.html' (UTF-8) -> 'index.html' (UTF-8)
> --2018-05-10 06:24:33--  http://google.com/
> Resolving google.com (google.com)... failed: Name or service not known.

nslookup bypasses system domain name resolver and querries DNS servers
directly comparing to wget or most of the other programs.

Do you experience the same issue with other programs that use system
resolver? E.g. "getent host google.com"? Maybe one of your name servers
misbehaves and only the second query to the second one succeeds. Maybe your
system resolved validates DNSSEC signatures and your network or name servers
block EDNS packets. What's your system resolver (/etc/nsswitch.conf), do you
use nscd or sssd caching deamons? If you do, what does happen if you flush
their caches (e.g. nscd --invalidate hosts). Capturing and studying network
packets while experiencing the issue would also help.

-- Petr


signature.asc
Description: PGP signature


Re: [Bug-wget] /usr/bin/env: invalid option -- 'S'

2019-05-31 Thread Petr Pisar
On Thu, May 30, 2019 at 09:56:33AM -0400, Jeffrey Walton wrote:
> I used PERL5LIB to put teests/ on path for Perl. It looks like at
> least one Debian machine I have is back to the Socket::inet_ntoa
> problems.
> 
> I'm calling it good.
> 
> The Perl people need to fix Socket::inet_ntoa, and the Debian people
> need to make it available. I'm guessing Debian is the holdup. They
> will leave things broke rather than supplying an update. It is a waste
> of time to file a Debian bug report.
> 
You can report your issues directly to Socket authors if you believe the issue
is not specific to Debian.

May I know what's your issue with Socket::inet_ntoa?

-- Petr


signature.asc
Description: PGP signature


Re: [Bug-wget] /usr/bin/env: invalid option -- 'S'

2019-06-02 Thread Petr Pisar
On Sun, Jun 02, 2019 at 02:00:54PM +0200, Tim Rühsen wrote:
> On 31.05.19 21:15, Petr Pisar wrote:
> > On Thu, May 30, 2019 at 09:56:33AM -0400, Jeffrey Walton wrote:
> >> I used PERL5LIB to put teests/ on path for Perl. It looks like at
> >> least one Debian machine I have is back to the Socket::inet_ntoa
> >> problems.
> >>
> >> I'm calling it good.
> >>
> >> The Perl people need to fix Socket::inet_ntoa, and the Debian people
> >> need to make it available. I'm guessing Debian is the holdup. They
> >> will leave things broke rather than supplying an update. It is a waste
> >> of time to file a Debian bug report.
> >>
> > You can report your issues directly to Socket authors if you believe the 
> > issue
> > is not specific to Debian.
> > 
> > May I know what's your issue with Socket::inet_ntoa?
> 
> It's not about Socket::inet_ntoa (sorry for not correcting this before).
> IMO, it's about https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=887590
> 
I see. Those are the IPv6 patches for HTTP::Daemon I wrote two years ago for
RHEL.

The patches are indeed quite large and have an effect on other packages that
use HTTP::Daemon. Especially on tests. Because various packages are not
prepared for HTTP::Daemon listening on an IPv6 socket. I understand why Debian
does not want to apply them to a stable distribution. Applying them would
change a behavior and people could get mad at them.

I can see two solutions for wget. Either use 127.0.0.1 instead of localhost
everywhere, or skip the particular test if HTTP::Daemon is unable to listen on
an IPv6 while plain Socket (or IO::Socket::IP) is.

-- Petr


signature.asc
Description: PGP signature