Re: [Bug-wget] Overly permissive hostname matching

2014-03-21 Thread Tim Ruehsen
On Thursday 20 March 2014 23:11:31 Daniel Stenberg wrote: On Thu, 20 Mar 2014, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl Ok, I'll be

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Tim Rühsen
Am Mittwoch, 19. März 2014, 10:59:05 schrieb Daniel Kahn Gillmor: I'm imagining a C library API that has a public suffix list context object that can do efficient lookups (however we define the lookups), and the library would bundle a pre-compiled context, based on the currently-known public

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 5:52 PM, Tim Rühsen tim.rueh...@gmx.de wrote: Am Mittwoch, 19. März 2014, 10:59:05 schrieb Daniel Kahn Gillmor: I'm imagining a C library API that has a public suffix list context object that can do efficient lookups (however we define the lookups), and the library

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Daniel Stenberg
On Thu, 20 Mar 2014, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl Ok, I'll be the first to rain on the parade. Sorry but it seems fit to do

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 6:11 PM, Daniel Stenberg dan...@haxx.se wrote: On Thu, 20 Mar 2014, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 23:11, Daniel Stenberg wrote: You do realize that with a *GPL license on the thing, you won't get adopted by OpenSSL, curl and possibly others? I can't prevent you of course and the decision is yours to make, but I'd prefer a BSD style license as then I could really consider

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 22:52, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl The first step was a psl_is_tld() function. There is a test case for some major

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 23:16, Jeffrey Walton wrote: I can't prevent you of course and the decision is yours to make, but I'd prefer a BSD style license as then I could really consider basing future enhancements of curl on this effort. Does GNU have a permissive license? I know permissive does not meet

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 7:11 PM, Ángel González keis...@gmail.com wrote: On 20/03/14 23:16, Jeffrey Walton wrote: I can't prevent you of course and the decision is yours to make, but I'd prefer a BSD style license as then I could really consider basing future enhancements of curl on this

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Daniel Stenberg
On Fri, 21 Mar 2014, Ángel González wrote: The LGPL would be an option. Not for curl though and probably not to other BSD/MIT licensed projects... -- / daniel.haxx.se

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 21/03/14 00:21, Daniel Stenberg wrote: On Fri, 21 Mar 2014, Ángel González wrote: The LGPL would be an option. Not for curl though and probably not to other BSD/MIT licensed projects... That's a good point. Jeff wrote: Isn't copyright assigned to GNU or FSF? No. By licensing

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 8:12 PM, Ángel González keis...@gmail.com wrote: On 21/03/14 00:21, Daniel Stenberg wrote: ... (Sorry, I don't know. I'm not a lawyer, so my solution is usually to avoid GPL code all together). That's a solution. Although it's a sad result from usage of a license

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Tue, 18 Mar 2014, Ángel González wrote: Daniel, how does cURL check correctness of the certificate hostname suffix? It insists on at least two dots. So yes, *.apple will cause problems for us too. I view the public suffix list as one of the worst kludges in networking history and while

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Wed, 19 Mar 2014, Daniel Kahn Gillmor wrote: It insists on at least two dots. So yes, *.apple will cause problems for us too. There are also errors in the opposite direction: it sounds like curl will accept a cert for *.co.uk, right? Exactly, due to the lack of public suffix awareness!

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 06:19 AM, Tim Ruehsen wrote: As a programmer, I want to have control. E.g. the option to load from a different file, or to switch off loading. Why ? e.g. for testing purposes, or simply imagine a swiss army knife client for experts - maybe they want to have control via CLI

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 10:59 AM, Daniel Kahn Gillmor d...@fifthhorseman.net wrote: On 03/19/2014 06:19 AM, Tim Ruehsen wrote: As a programmer, I want to have control. E.g. the option to load from a different file, or to switch off loading. Why ? e.g. for testing purposes, or simply imagine a

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with ! That sounds wrong: A rule may begin with a ! (exclamation mark). If it does, it is labelled as a exception rule and then treated as if the exclamation mark is not present. -- / daniel.haxx.se

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:37 AM, Jeffrey Walton noloa...@gmail.com wrote: On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg dan...@haxx.se wrote: On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with ! That sounds wrong: A rule may begin with a ! (exclamation mark).

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg dan...@haxx.se wrote: On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with ! That sounds wrong: A rule may begin with a ! (exclamation mark). If it does, it is labelled as a exception rule and then treated as if

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:45 AM, Jeffrey Walton noloa...@gmail.com wrote: On Wed, Mar 19, 2014 at 11:37 AM, Jeffrey Walton noloa...@gmail.com wrote: On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg dan...@haxx.se wrote: On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 11:55 AM, Jeffrey Walton wrote: Also, be careful of where you are pulling the list from. I got burned by pulling a list that was not being updated (https://bugzilla.mozilla.org/show_bug.cgi?id=968064). i've been similarly burned before too, but i settled on the mxr address i just

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Ángel González
On 19/03/14 16:37, Jeffrey Walton wrote: On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenbergdan...@haxx.se wrote: On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with ! That sounds wrong: A rule may begin with a ! (exclamation mark). If it does, it is labelled as

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 3:03 PM, Ángel González keis...@gmail.com wrote: On 19/03/14 16:37, Jeffrey Walton wrote: ... Also note that by removing the *. from the beginning of the lines*, you are acepting more hosts than you should, such as a certificate for *.com.bd (represented as *.bd in

[Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Jeffrey Walton
I believe wget has a security flaw in its certificate hostname matching code. In the attached server certificate, the hostname is provided via a Subject Alt Name (SAN). The only SAN entry is a DNS name for *.com. Also attached is the default CA, which was used to sign the server's certificate.

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Daniel Kahn Gillmor
Hi Jeffrey-- On 03/18/2014 01:43 AM, Jeffrey Walton wrote: I believe wget has a security flaw in its certificate hostname matching code. In the attached server certificate, the hostname is provided via a Subject Alt Name (SAN). The only SAN entry is a DNS name for *.com. Also attached is

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Ángel González
I don't think wget should be checking correct hostname scope of the certificate. I mean, it'd be ok to have some general rule as noone can use a certificate for *.whatever or *. [1] but embedding the Public Suffix List seems overkill. And the implementation should probably be performed at

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Tim Rühsen
Hi Jeffrey, thanks for pointing this out. BTW, to reproduce the issue I used a GnuTLS compiled/linked version of Wget: $ wget -d --ca-certificate=ca-rsa-cert.pem --private-key=ca-rsa-key-plain.pem https://example.com:8443 2014-03-18 21:48:04 (1.88 GB/s) - Read error at byte 5116 (The TLS

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Daniel Kahn Gillmor
On 03/18/2014 05:31 PM, Tim Rühsen wrote: $ wget -d --ca-certificate=ca-rsa-cert.pem --private-key=ca-rsa-key-plain.pem https://example.com:8443 2014-03-18 21:48:04 (1.88 GB/s) - Read error at byte 5116 (The TLS connection was non-properly terminated.).Retrying. There seems to be a