Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[Ethan Welker]

Follow-up Comment #6, bug #51666 (project wget):

Thank you for your feedback.

Regarding the salt: I used one because OpenSSH uses one and that was both what
the original bug report references, and written by somebody likely more
experienced than me Re: security. If you want me to remove the salting or
something I can do so.

This version of the patch (only one this time, because there's little point in
only separating adding the tests) has the following changes from the previous
patchset:

* Uses gnulib hashing functions instead of gnutls/openssl ones (Sorry, I
didn't know gnulib had those!).
* Fixes the interleaved versions duplicate entry thing.
* Adds dependency on gnulib's crypto/gc and crypto/gc-random (not any
crypto/gc-) to generate decent quality randomness for the
salt. Seeing as the salt apparently isn't that important, I guess it could be
switched to random_number if this is a problem.
* In case someone does want to switch to crypto/gc- in the
future (to easily add another hashing algorithm or something), the prefix was
switched from 1 to 5 to match the value of GC_SHA256 in gc.h.

(file #44848)
___

Additional Item Attachment:

File name: 0001-Hash-domains-in-HSTS-database-to-improve-privacy.patch Size:25
KB


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[Tim Ruehsen]

Follow-up Comment #5, bug #51666 (project wget):

Thanks for addressing the issue.

Saving the salt together with the (salted) hash isn't of big help when we talk
about a limited set of input strings. You can get complete lists of existing
domains and brute force through them in a few seconds. Can even be optimized
by starting with the top 1m domains. I just mention this to make clear that
this way of obscuring is far from being safe. It is just slightly more effort
to reverse the domain names in comparison to unsalted hashes.

Anyways, it helps from being fly-by looked at, e.g. on the console.

I would like to ask you to not use OpenSSL for hashing. We have/use the SHA256
digest functions from gnulib anyways. So it should be straight forward.


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[Ethan Welker]

Follow-up Comment #4, bug #51666 (project wget):

Hello!

I've written a series of patches (attached) that adds this feature. Please let
me know if there's some other way that I should be submitting these instead.

Some notes:

* I have little security experience, so these should be checked over somewhat
carefully.
* The OpenSSL implementation of ssl_hash_salt uses functions only in OpenSSL
0.9.7 and later. I'm not familiar enough with OpenSSL to get it to support
anything prior to that, but the rest of openssl.c seems to imply that some
versions before that are supported. Sorry about this!
* This patch seamlessly and automatically converts old hsts databases to
hashed databases. Unpatched versions of Wget parse those databases without
issue, though they functionally ignore any converted entry.
** If a patched Wget creates a hashed entry for a HSTS-enabled website, then
an unpatched Wget creates a unhashed entry for the same website, then a
patched Wget converting that entry will create a duplicate entry in the store.
I'm not sure how much of a concern this is.
* This switches the HSTS store from a hash table to an array, and each search
for a host creates a hash for each element in that array until either one
matches or the end of the array is reached. This isn't a problem for
reasonably sized stores; in my test on a 50-entry .wget-hsts, an unpatched
wget was 100 ms _slower_ (presumably due to network jitter). However I can
imagine that if you have several thousand entries this could become a
noticeable slowdown.

(file #44842, file #44843, file #44844)
___

Additional Item Attachment:

File name: 0001-Add-utility-functions-for-hashing-and-salting.patch Size:4 KB
File name: 0002-src-hsts.c-Hash-HSTS-enabled-domains-to-improve-priv.patch
Size:23 KB
File name: 0003-src-hsts.c-Add-test-for-hashed-domains-stored-in-dat.patch
Size:2 KB


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[Tim Rühsen]

On Freitag, 18. August 2017 14:51:12 CEST Ander Juaristi wrote:
> Follow-up Comment #2, bug #51666 (project wget):
> 
> I'm not generally against these kind of small tweaks that don't harm and
> slightly improve user's privacy.
> 
> If Firefox doesn't do it, we don't care: it's their business and they will
> end up doing it if users request that feature (maybe because they saw it in
> wget).
> 
> Private SSH keys can be protected with a password if you want to.

As long as it is optional...

It would be nice being file compatible with Firefox (at least reading Firefox 
HSTS db).
Maybe the sqlite backend that has been mentioned earlier should then work with 
the same settings (hashed/not hashed).

> We can do both, hash and still keep the readable to the user only. If the
> overhead is not much I would go for it. That is the basis of every security
> framework out there: if the benefits of having 2 security mechanisms instead
> of only 1 outweigh the drawbacks, then implement 2 instead of 1.

Absolutely, but in this special case you open up a can of worms. From a 
security standpoint, the average home directory is a nightmare. Once someone 
gets access to it (read or write)...

Regards, Tim

signature.asc
Description: This is a digitally signed message part.

Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[Tim Ruehsen]

Follow-up Comment #1, bug #51666 (project wget):

The files are read/write only to the user (in case you didn't protect your
home directory appropriately).

If you are still concerned, what about
~/.mozilla/firefox/*.default/SiteSecurityServiceState.txt and likely hundreds
or thousands of other files containing potential private information ?

Not thinking of your private keys lying around in .ssh/.

And even file dates and sizes are potential private data leaks.

While we could hash anything, it would be way safer for you to protect your
complete home directory.

We could think about an option, just in case you want to keep your HSTS cache
in a public space, e.g. /var. But we already think of using SQLite for storing
and maybe sharing with Firefox.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

Bug#870813: [bug #51666] Please hash the hostname in ~/.wget-hsts files

[NoëlKöthe]

URL:
  

 Summary: Please hash the hostname in ~/.wget-hsts files
 Project: GNU Wget
Submitted by: nok
Submitted on: Sat 05 Aug 2017 04:14:03 PM CEST
Category: Feature Request
Severity: 3 - Normal
Priority: 5 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
 Originator Name: 
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
 Release: trunk
Operating System: GNU/Linux
 Reproducibility: Every Time
   Fixed Release: None
 Planned Release: None
  Regression: No
   Work Required: None
  Patch Included: No

___

Details:

Hello,

a feature request from a Debian user:

--8<--
Hi,

I recently discovered the .wget-hsts file in my home directory which is used
to persist state for HSTS:

  https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security

It contains lines such as:

  # HSTS 1.0 Known Hosts database for GNU Wget.
  # Edit at your own risk.
  #  
  github.com0   1   1450887745  31536000
  ftp-master.debian.org 0   0   1472482586  15552000
  diffoscope.org0   0   1449765396  15768000
  reproducible-builds.org   0   0   1471542629  15552000
  www.dropbox.com  0   1   1458394011  15552000
  reproducible.debian.net   0   0   1448074844  15552000
  […]

I can't help but think of ~/.ssh/known_hosts which moved to hashing the
hostname for various security/privacy concerns. Shouldn't wget make the
parallel change?
--8<--
https://bugs.debian.org/870813

Thank you.




___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/