Re: [Bug-wget] Fixing progress bar assertions in multibyte locales

[Giuseppe Scrivano]

Darshit Shah  writes:

> Hi everyone,
>
> As mentioned earlier, there was a bug in the progress bar
> implementation that caused the tests in multi-byte and multi-column
> locales to fail. I've rectified the issue by re-working each part of
> the progress bar to ensure that they are indeed using exactly the
> amount of space that has been allocated.
>
> I also tried to clean up some of the code. I tested these commits on
> Travis and there seem to be no issues. All tests are passing with
> Russian, Turkish and C locales.
>
> Please review the patch, and if good, I'll push it to master. Since it
> touches the most user-centric part of Wget, I'd like to see it as well
> reviewed and tested as possible.

I have looked at the patches and they seem fine to me.  Feel free to
push them.

Thanks,
Giuseppe

Re: [Bug-wget] Marking Release v1.17.1?

[Tim Ruehsen]

The regression fix has already been pushed.

Tim

On Wednesday 09 December 2015 08:44:28 Darshit Shah wrote:
> On 12/09, Giuseppe Scrivano wrote:
> >Darshit Shah  writes:
> >> With my last set of patches, we have fixed all the issues reported /
> >> identified after the 1.17 release. Hence, maybe we should consider
> >> releasing a 1.17.1 bugfix release?
> >
> >yes, we should do that.  I can tag a new release tomorrow or on Friday.
> 
> There's one patch by Tim that's pending and my fixes for the progress bar.
> We should get them in before tagging.
> 
> >Regards,
> >Giuseppe

Re: [Bug-wget] wget 1.17 does not use .netrc for http auth

[Tim Ruehsen]

Hi Darshit,

the fix is already in master.

Tim

On Tuesday 08 December 2015 23:06:34 Darshit Shah wrote:
> Axel hasn't responded, but this is a valid fix. Tim could you please push it
> in time for the bug fix release?
> 
> On 11/23, Tim Rühsen wrote:
> >Thanks Axel !
> >
> >You are right, it is a regression introduced ~ half a year ago.
> >
> >Could you try this patch ?
> >
> >Tim
> >
> >On Monday 23 November 2015 15:16:41 Axel Reinhold wrote:
> >> Dear wget-Maintainers,
> >> 
> >> wget-1.17 does not more use .netrc for getting http-auth-info
> >> this is a regression i think.
> >> 
> >> ==
> >> with wget-1.17:
> >> ==
> >> $ /opt/wget/bin/wget -v -O- http://calea.wpack.de/sites/active
> >> ==
> >> --2015-11-23 15:11:05--  http://calea.wpack.de/sites/active
> >> Resolving calea.wpack.de (calea.wpack.de)... 188.138.31.224
> >> Connecting to calea.wpack.de (calea.wpack.de)|188.138.31.224|:80...
> >> connected. HTTP request sent, awaiting response... 401 Unauthorized
> >> Username/Password Authentication Failed.
> >> ==
> >> $ /opt/wget/bin/wget --http-user=rcon --http-password=wefhhoG -O-
> >> http://calea.wpack.de/sites/active
> >> ==
> >> --2015-11-23 15:11:24--  http://calea.wpack.de/sites/active
> >> Resolving calea.wpack.de (calea.wpack.de)... 188.138.31.224
> >> Connecting to calea.wpack.de (calea.wpack.de)|188.138.31.224|:80...
> >> connected. HTTP request sent, awaiting response... 401 Unauthorized
> >> Authentication selected: Digest realm="wpack",
> >> nonce="a1LgzDUlBQA=54905863b96f04dca4ae0810eb594fdcf4066221",
> >> algorithm=MD5, qop="auth" Reusing existing connection to
> >> calea.wpack.de:80.
> >> HTTP request sent, awaiting response... 200 OK
> >> Length: 146 [text/html]
> >> Saving to: 'STDOUT'
> >> -  0%[
> >> 
> >>   ]   0  --.-KB/s 1 city7
> >>   201511231325
> >> 
> >> city7OKwp 212.114.252.79 City-Hotel München ***Superior 7 bambini
> >> 201511231326 Juli24Mg 84.57.88.108 Bambini Kinderwelt GmbH -
> >> 
> >> 100%[>]
> >> 146
> >> 
> >>  --.-KB/s   in 0s 2015-11-23 15:11:24 (15.5 MB/s) - written to stdout
> >> 
> >> [146/146]
> >> ==
> >> 
> >> ==
> >> with wget-1.16.3:
> >> ==
> >> $ /opt/wget/bin/wget -v -O- http://calea.wpack.de/sites/active
> >> ==
> >> --2015-11-23 15:14:01--  http://calea.wpack.de/sites/active
> >> Resolving calea.wpack.de (calea.wpack.de)... 188.138.31.224
> >> Connecting to calea.wpack.de (calea.wpack.de)|188.138.31.224|:80...
> >> connected. HTTP request sent, awaiting response... 401 Unauthorized
> >> Authentication selected: Digest realm="wpack",
> >> nonce="fnlD1jUlBQA=cbebe19e1ffead899e352fb4e77f5fd8d64803f6",
> >> algorithm=MD5, qop="auth" Reusing existing connection to
> >> calea.wpack.de:80.
> >> HTTP request sent, awaiting response... 200 OK
> >> Length: 146 [text/html]
> >> Saving to: 'STDOUT'
> >> -  0%[
> >> 
> >>   ]   0  --.-KB/s 1 city7
> >>   201511231325
> >> 
> >> city7OKwp 212.114.252.79 City-Hotel München ***Superior 7 bambini
> >> 201511231326 Juli24Mg 84.57.88.108 Bambini Kinderwelt GmbH -
> >> 
> >> 100%[>]
> >> 146
> >> 
> >>  --.-KB/s   in 0s 2015-11-23 15:14:01 (13.9 MB/s) - written to stdout
> >> 
> >> [146/146]
> >> ==
> >
> >From a1da7fd960dba9e733e36a47bf144827c7d0e154 Mon Sep 17 00:00:00 2001
> >From: =?UTF-8?q?Tim=20R=C3=BChsen?= 
> >Date: Mon, 23 Nov 2015 17:50:59 +0100
> >Subject: [PATCH] Fix regression in HTTP authentication
> >
> >* src/http.c (initialize_request): Fix wrong params to search_netrc()
> >
> >Regression introduced in commit 29850e77
> >---
> >
> > src/http.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> >diff --git a/src/http.c b/src/http.c
> >index 9d71483..8916d2b 100644
> >--- a/src/http.c
> >+++ b/src/http.c
> >@@ -1872,7 +1872,7 @@ initialize_request (struct url *u, struct http_stat
> >*hs, int *dt, struct url *pr>
> >   /* Find the username and password for authentication. */
> >   *user = u->user;
> >   *passwd = u->passwd;
> >
> >-  search_netrc (u->host, (const char **), (const char **), 0);
> >+  

Re: [Bug-wget] Fwd: New Defects reported by Coverity Scan for GNU Wget

[Juaristi Álamos , Ander]

Darshit, could you test if these fixes pass the Coverity tests?
I'm not particularly sure of the HSTS fix.

Regards,
- AJ

On Sun, 2015-12-06 at 22:45 +0100, Darshit Shah wrote:
> -- Forwarded message --
> From:  
> Date: 6 December 2015 at 22:39
> Subject: New Defects reported by Coverity Scan for GNU Wget
> To: dar...@gmail.com
> 
> 
> 
> Hi,
> 
> Please find the latest report on new defect(s) introduced to GNU Wget
> found with Coverity Scan.
> 
> 6 new defect(s) introduced to GNU Wget found with Coverity Scan.
> 
> 
> New defect(s) Reported-by: Coverity Scan
> Showing 6 of 6 defect(s)
> 
> 
> ** CID 1341706:(RESOURCE_LEAK)
> /src/ftp.c: 1518 in getftp()
> /src/ftp.c: 1528 in getftp()
> /src/ftp.c: 1518 in getftp()
> /src/ftp.c: 1518 in getftp()
> 
> 
> 
> *** CID 1341706:(RESOURCE_LEAK)
> /src/ftp.c: 1518 in getftp()
> 1512 logputs (LOG_NOTQUIET, "Server does not want to
> resume the SSL session. Trying with a new one.\n");
> 1513   if (!ssl_connect_wget (dtsock, u->host, NULL))
> 1514 {
> 1515   fd_close (csock);
> 1516   fd_close (dtsock);
> 1517   logputs (LOG_NOTQUIET, "Could not perform SSL
> handshake.\n");
> >>> CID 1341706:(RESOURCE_LEAK)
> >>> Variable "fp" going out of scope leaks the storage it points to.
> 1518   return CONERROR;
> 1519 }
> 1520 }
> 1521   else
> 1522 logputs (LOG_NOTQUIET, "Resuming SSL session in data
> connection.\n");
> 1523
> /src/ftp.c: 1528 in getftp()
> 1522 logputs (LOG_NOTQUIET, "Resuming SSL session in data
> connection.\n");
> 1523
> 1524   if (!ssl_check_certificate (dtsock, u->host))
> 1525 {
> 1526   fd_close (csock);
> 1527   fd_close (dtsock);
> >>> CID 1341706:(RESOURCE_LEAK)
> >>> Variable "fp" going out of scope leaks the storage it points to.
> 1528   return CONERROR;
> 1529 }
> 1530 }
> 1531 #endif
> 1532
> 1533   /* Get the contents of the document.  */
> /src/ftp.c: 1518 in getftp()
> 1512 logputs (LOG_NOTQUIET, "Server does not want to
> resume the SSL session. Trying with a new one.\n");
> 1513   if (!ssl_connect_wget (dtsock, u->host, NULL))
> 1514 {
> 1515   fd_close (csock);
> 1516   fd_close (dtsock);
> 1517   logputs (LOG_NOTQUIET, "Could not perform SSL
> handshake.\n");
> >>> CID 1341706:(RESOURCE_LEAK)
> >>> Variable "fp" going out of scope leaks the storage it points to.
> 1518   return CONERROR;
> 1519 }
> 1520 }
> 1521   else
> 1522 logputs (LOG_NOTQUIET, "Resuming SSL session in data
> connection.\n");
> 1523
> /src/ftp.c: 1518 in getftp()
> 1512 logputs (LOG_NOTQUIET, "Server does not want to
> resume the SSL session. Trying with a new one.\n");
> 1513   if (!ssl_connect_wget (dtsock, u->host, NULL))
> 1514 {
> 1515   fd_close (csock);
> 1516   fd_close (dtsock);
> 1517   logputs (LOG_NOTQUIET, "Could not perform SSL
> handshake.\n");
> >>> CID 1341706:(RESOURCE_LEAK)
> >>> Variable "fp" going out of scope leaks the storage it points to.
> 1518   return CONERROR;
> 1519 }
> 1520 }
> 1521   else
> 1522 logputs (LOG_NOTQUIET, "Resuming SSL session in data
> connection.\n");
> 1523
> 
> ** CID 1341705:  Security best practices violations  (TOCTOU)
> /src/hsts.c: 479 in hsts_store_open()
> 
> 
> 
> *** CID 1341705:  Security best practices violations  (TOCTOU)
> /src/hsts.c: 479 in hsts_store_open()
> 473
> 474   if (file_exists_p (filename))
> 475 {
> 476   if (stat (filename, ) == 0)
> 477 store->last_mtime = st.st_mtime;
> 478
> >>> CID 1341705:  Security best practices violations  (TOCTOU)
> >>> Calling function "fopen" that uses "filename" after a check function. 
> >>> This can cause a time-of-check, time-of-use race condition.
> 479   fp = fopen (filename, "r");
> 480   if (!fp || !hsts_read_database (store, fp, false))
> 481 {
> 482   /* abort! */
> 483   hsts_store_close (store);
> 484   xfree (store);
> 
> ** CID 1273467:  API usage errors  (BUFFER_SIZE)
> /lib/md5.c: 291 in md5_process_bytes()
> 
> 
> 
> *** CID 1273467:  API usage errors  (BUFFER_SIZE)
> /lib/md5.c: 291 

Re: [Bug-wget] --no-check-cert does not avoid cert warning

[Ángel González]

On 02/12/15 14:33, Giuseppe Scrivano wrote:

Tim agreed so I've changed the patch to add --check-certificate=quiet.

How does the new version look?

Thanks,
Giuseppe

diff --git a/doc/wget.texi b/doc/wget.texi
index c647e33..9cc2bb2 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -1725,6 +1725,9 @@ this option to bypass the verification and proceed with 
the download.
  site's authenticity, or if you really don't care about the validity of
  its certificate.}  It is almost always a bad idea not to check the
  certificates when transmitting confidential or important data.
+If you are really sure of what you are doing, you can specify
+--check-certificate=quiet to ask wget to not print any warning about
+invalid certificates, in most cases this is the wrong thing to do.


What about adding a hint about providing the self-signed as trusted?

Something like:

diff --git a/doc/wget.texi b/doc/wget.texi
index 64cb056..f3925ca 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -1725,9 +1725,12 @@ this option to bypass the verification and 
proceed with the download.

 site's authenticity, or if you really don't care about the validity of
 its certificate.}  It is almost always a bad idea not to check the
 certificates when transmitting confidential or important data.
-If you are really sure of what you are doing, you can specify
---check-certificate=quiet to tell wget to not print any warning about
-invalid certificates, in most cases this is the wrong thing to do.
+For self-signed/internal certificates, you should download the 
certificate

+and verify against that instead of forcing this insecure mode.
+If you are really sure of not desiring any certificate verification, you
+can specify --check-certificate=quiet to tell wget to not print any
+warning about invalid certificates, albeit in most cases this is the
+wrong thing to do.

 @cindex SSL certificate
 @item --certificate=@var{file}


Regards

[Bug-wget] Broken: darnir/wget#58 (darnir_testing - 1b9244e)

[Travis CI]

Build Update for darnir/wget
-

Build: #58
Status: Broken

Duration: 5 minutes and 20 seconds
Commit: 1b9244e (darnir_testing)
Author: Darshit Shah
Message: Improve travis script runtime

* contrib/travis-ci: Run tests in multiple languages only when all
features are turned on. This tests the maximum number of strings. For
the rest, testing only in C locale is enough.

View the changeset: 
https://github.com/darnir/wget/compare/b31273affbe0...1b9244e791e9

View the full build log and details: 
https://travis-ci.org/darnir/wget/builds/95939218

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Re: [Bug-wget] Windows cert store support

[Ángel González]

On 09/12/15 03:06, Random Coder wrote:

I'm not sure if the wget maintainers would be interested, but I've
been carrying this patch around in my private builds of wget for a
while.  It allows wget to load SSL certs from the default Windows cert
store.

The patch itself is fairly straightforward, but as it changes the
default SSL behavior, and no care was taken to follow coding convents
when I wrote it, so it's probably not ready for inclusion in the
codebase.  Still, if it's useful, feel free to use it for ideas.

Wow, supporting the OS store would certainly be very cool.

I would probably move it to windows.c and attempt to make it also work 
in gnutls, but in general it looks good.


Thanks!

Re: [Bug-wget] Windows cert store support

[Random Coder]

On Wed, Dec 9, 2015 at 4:12 PM, Ángel González  wrote:
> I would probably move it to windows.c and attempt to make it also work in
> gnutls, but in general it looks good.

Fair enough.  I'll fix up the patch in the coming weeks.  If anyone
else wants a stab before me, feel free!