Re: Q: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

TL;DR: Any encrypted transport protocol (e.g. https) breaks caching.

On Sat, Aug 21, 2021 at 10:40:32AM +0200, Wouter Verhelst wrote:
> This could be useful for both the "I've got a slow uplink and would like
> it to not be overwhelmed at the BSP I'm hosting for my Debian friends"
> type as well as the "I'm an ISP and I want to provide a mirror to Debian
> users so we can reduce our uplink connection a bit" type of situations.

Yes, please. Another situation is: "My home houses around 10 Debian
machines, some of which are mobile."

I'm actually doing this since around 10 years.

My first implementation (predating both auto-apt-proxy and
squid-deb-proxy-client) was a custom DHCP option containing a proxy.
I've locally used that with success for quite a while in my own
environments. It does have a few downsides:
 * Integrating it with isc-dhcp-client was impossible in a
   policy-compliant way.
 * IPv4 only.
 * As others have pointed out: Breaks with https.

Other commenters mentiond squid-deb-proxy-client using
_apt_proxy._tcp. via avahi.

auto-apt-proxy has a few techniques beyond that.

Unfortunately, none of them are enabled by default, so in general it
doesn't just work.

In any case, I for one would actively opt out of https, because it would
make apt unusable for me. My cache has a byte hit rate around 90% - 95%.
It seems to deliver around 4TB of .debs and stuff per year here. How
would you get that through DSL? Using https would make apt unbearably
slow for me.

Honouring _apt_proxy._tcp. by default would be cool indeed.
apt-cacher-ng automatically publishes this as does squid-deb-proxy.
However doing so would completely break if we were to go https by
default.

Helmut

Re: Q: Use https for {deb,security}.debian.org by default

[Julien Cristau]

On Sat, Aug 21, 2021 at 10:28:04AM +0200, Wouter Verhelst wrote:
> On Thu, Aug 19, 2021 at 10:11:33PM +, Jeremy Stanley wrote:
> > On 2021-08-19 16:37:13 -0400 (-0400), Kyle Edwards wrote:
> > > On 8/19/21 3:46 PM, Simon Richter wrote:
> > > > For the most part, users would configure https if they are behind a
> > > > corporate firewall that disallows http, or modifies data in-flight so
> > > > signature verification fails, everyone else is better off using plain
> > > > http.
> > > 
> > > Or they might configure https on the sheer principle of not wanting to 
> > > have
> > > their traffic hoovered up by their ISP or anyone else who might be
> > > listening.
> > 
> > While this does complicate it, a snooping party can still know the
> > site they're connecting to via SNI happening unencrypted,
> 
> SNI is not unencrypted if you do TLS1.3...
> 
It is, though...  ECH (née ESNI)
https://datatracker.ietf.org/doc/draft-ietf-tls-esni/ is still WIP.

Cheers,
Julien

Re: Q: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Sat, Aug 21, 2021 at 11:05:23PM +, Stephan Verbücheln wrote:
> What about HTTP 304 Not Modified?

What about them? Care to give details?


Note that APT nowadays hardly makes requests which can legally be
replied to with 304 as it knows which index files changed (or not)
based on comparing the old and new Release files.

That leaves the Release file itself, which even if the server replied
304 undergoes again the signature and other consistency checks
– including Valid-Until. Not only to detect serious attacks, but also to
detect if a mirror is no longer synced as the most common form of 'man
in the middle' "attack" https has no chance of preventing or detecting.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Stephan Verbücheln]

What about HTTP 304 Not Modified?

Regards

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-21 12:04:32 +0100 (+0100), Phil Morrell wrote:
> On Sat, Aug 21, 2021 at 10:40:32AM +0200, Wouter Verhelst wrote:
[...]
> > However, I've not been able to come up with a scheme which is simple
> > enough to be doable on a LAN while at the same time be usable by larger
> > network providers, *and* which can't also be abused by MitM attackers.
> 
> Isn't the MitM handled by archive signatures etc., hence why http is
> fine? True I haven't tested this in a large network, since usually
> configuration management is in place, but apparently mDNS can even
> traverse routers via Multicast BGP.

As already pointed out by others, the risk is that a MitM could
serve you outdated package indices and packages, silently blocking
you from patching some known vulnerability until the index expires,
which might provide the attacker some extra time to work on
exploiting that vulnerability. The practicality of this particular
attack isn't all that high, as there are often going to be other
avenues of compromise which involve less effort on the part of the
attacker anyway. Still, people are correct to call it out as some
form of risk.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Sat, Aug 21, 2021 at 09:45:54AM +0200, Tomas Pospisek wrote:
> On 21.08.21 09:14, Philipp Kern wrote:
> > defense in depth if we wanted to, but maybe the world just agreed that
> > you need to get your clock roughly correct. ;-)
> 
> I remember seeing apt-get refusing to update packages or the index because
> of them "having timestamps in the future" or in other words system time
> being out of sync in direction of the past.

APT requires the time to be more or less correct since ever¹ by virtue
of e.g. gpg keys (or signatures) expiring and expired keys are bad.

In recent years we became more reliant on the time to ensure
repositories are somewhat current refusing repos from too long in the
past as well as from the future. At least these can be worked around
with -o Acquire::Check-Date=false.

For gpg you will need another workaround I can't remember of the top of
my hat. There are likely more problems as it is easier to just set the
clock approximately correct than to remember all the workarounds in
"the time of need"…


Best regards

David Kalnischkies

¹ okay, ~15 years of apt-secure are not exactly ever, but close enough.


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Sat, Aug 21, 2021 at 12:04:32PM +0100, Phil Morrell wrote:
> On Sat, Aug 21, 2021 at 10:40:32AM +0200, Wouter Verhelst wrote:
> > On Fri, Aug 20, 2021 at 07:20:22PM +, Jeremy Stanley wrote:
> > > Yes transparent proxies or overridden DNS lookups could be used to
> > > direct deb.debian.org and security.debian.org to your alternative
> > > location,
> > 
> > I've been thinking for a while that we should bake a feature in apt
> > whereby a network administrator can indicate somehow that there is a
> > local apt mirror and that apt should use that one in preference to
> > deb.debian.org.
> 
> This already exists in the form of an avahi service announcement for
> _apt_proxy._tcp, issued by both squid-deb-proxy and apt-cacher-ng.
> Literally the only thing needed client-side is installation of
> squid-deb-proxy-client […]

That will instruct apt to use the proxy to connect to the internet, but
this is quite literal in meaning: apt will perform a CONNECT request
establishing a tunnel between itself and the remote server via the
proxy effectively by-passing any functionality the proxy would provide
if we wouldn't connect to the remote with https (as with http apt would
just issue GET requests to the proxy it could interact with).


apt can't just downgrade https to http if it knows about a proxy,
especially if that knowledge is provided by external potentially
untrusted sources. To do that we would need to at least ask the user
interactively if its okay to send the requests unencrypted to the proxy.

There is precedence with cdrom asking the user interactively to change
CDs if needed, so it isn't an entirely new concept, but libapt has no
generic question-asking code and cdrom is a cake walk compared to the
monster that is our http(s) implementation, so that is still a non-
trivial amount of code someone would need to write. Also in the libapt
front ends as you still need at least a bit of UI to actually expose the
question to the user.


Depending on how much control you have over the clients it might be
a lot easier to work with the mirror method. It can be (ab)used for
a lot more than most people give it credit for (Disclaimer: As I wrote
the current incarnation, I might be a *tiny bit* biased). That isn't
helping of course if you have no control at all over the clients as you
need some form of opt in at least. So far, that opt in was using http.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Thomas Goirand]

On 8/20/21 4:56 PM, Russ Allbery wrote:
> Jeremy Stanley  writes:
> 
>> I agree with all of the above, my point was that the current state of
>> HTTPS doesn't especially improve integrity for Debian package management
>> over the signed indices and checksums we already rely on, and trying to
>> use HTTPS for privacy/secrecy (which isn't really what it was designed
>> for) is still and perhaps even increasingly misguided. Of course lots of
>> people will continue to expect magic HTTPS fairy dust to protect them
>> and ward off evil, but the only legitimate reason I can see for Debian
>> changing the default protocol for sources.list entries is to avoid
>> having to pointlessly debate the minimal benefits of HTTPS with people
>> who drink whatever cool-aid they're told by security "experts" (HTTP
>> bad, HTTPS good, drink up!).
> 
> Do you think using HTTPS makes security worse?

Doing an update over Tor would be a much nicer improvement (so that
nobody can tell what package you didn't upgrade yet...).

> Personally, I think we should switch our default to HTTPS not because we
> have a specific security flaw in mind against which HTTPS provides some
> protection but because it's consistent with the general message that a lot
> of us (including, for example, the EFF and the IETF) are trying to send to
> average users who don't have the expertise to analyze any of this: use TLS
> by default wherever you can.  It's not a panacea, but ubiquitous, default
> use of TLS helps both your security and your privacy compared to either
> the previous default of no TLS or spending a bunch of mental energy
> picking and choosing.
> 

IMO, it's consistent with blindly trusting HTTPS which isn't helping
much here (especially compared to Tor), and if we want to promote
something, that'd be Tor, not the blindly trusted bunch of CAs...

Cheers,

Thomas Goirand (zigo)

Re: Q: Use https for {deb,security}.debian.org by default

[Phil Morrell]

On Sat, Aug 21, 2021 at 10:40:32AM +0200, Wouter Verhelst wrote:
> On Fri, Aug 20, 2021 at 07:20:22PM +, Jeremy Stanley wrote:
> > Yes transparent proxies or overridden DNS lookups could be used to
> > direct deb.debian.org and security.debian.org to your alternative
> > location,
> 
> I've been thinking for a while that we should bake a feature in apt
> whereby a network administrator can indicate somehow that there is a
> local apt mirror and that apt should use that one in preference to
> deb.debian.org.

This already exists in the form of an avahi service announcement for
_apt_proxy._tcp, issued by both squid-deb-proxy and apt-cacher-ng.
Literally the only thing needed client-side is installation of
squid-deb-proxy-client, which is also available in udeb form implying
that d-i already uses it.

> This could be useful for both the "I've got a slow uplink and would like
> it to not be overwhelmed at the BSP I'm hosting for my Debian friends"
> type as well as the "I'm an ISP and I want to provide a mirror to Debian
> users so we can reduce our uplink connection a bit" type of situations.

It's a great solution for everyone on the same wifi network, if everyone
has squid-deb-proxy-client installed then just one person can spawn a
proxy and suddenly everyone's caching through them.

> However, I've not been able to come up with a scheme which is simple
> enough to be doable on a LAN while at the same time be usable by larger
> network providers, *and* which can't also be abused by MitM attackers.

Isn't the MitM handled by archive signatures etc., hence why http is
fine? True I haven't tested this in a large network, since usually
configuration management is in place, but apparently mDNS can even
traverse routers via Multicast BGP.


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Philip Hands]

Wouter Verhelst  writes:

> On Fri, Aug 20, 2021 at 07:20:22PM +, Jeremy Stanley wrote:
>> Yes transparent proxies or overridden DNS lookups could be used to
>> direct deb.debian.org and security.debian.org to your alternative
>> location,
>
> I've been thinking for a while that we should bake a feature in apt
> whereby a network administrator can indicate somehow that there is a
> local apt mirror and that apt should use that one in preference to
> deb.debian.org.
>
> This could be useful for both the "I've got a slow uplink and would like
> it to not be overwhelmed at the BSP I'm hosting for my Debian friends"
> type as well as the "I'm an ISP and I want to provide a mirror to Debian
> users so we can reduce our uplink connection a bit" type of situations.
>
> However, I've not been able to come up with a scheme which is simple
> enough to be doable on a LAN while at the same time be usable by larger
> network providers, *and* which can't also be abused by MitM attackers.

We could declare that if one can find a TXT record in the local domain
(e.g. _DEBIAN_LOCAL_ARCHIVE.example.com) then one should use its
contents in order to configure an additional source for packages, such
that one gets the signed Release file from one's normally configured
sources, and then when getting subsequent files gives the local source a
try, falling back to the normal setup when downloads or checksums fail.

I can see that one could try a DoS of sorts by setting up the TXT record
to point at a tarpit, say, but that could be handled by setting short
timeouts, and giving up on the local server after some number of failures.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY

Re: Q: Use https for {deb,security}.debian.org by default

[Tobias Frost]

On Sat, Aug 21, 2021 at 10:40:32AM +0200, Wouter Verhelst wrote:
> On Fri, Aug 20, 2021 at 07:20:22PM +, Jeremy Stanley wrote:
> > Yes transparent proxies or overridden DNS lookups could be used to
> > direct deb.debian.org and security.debian.org to your alternative
> > location,
> 
> I've been thinking for a while that we should bake a feature in apt
> whereby a network administrator can indicate somehow that there is a
> local apt mirror and that apt should use that one in preference to
> deb.debian.org.
>
> This could be useful for both the "I've got a slow uplink and would like
> it to not be overwhelmed at the BSP I'm hosting for my Debian friends"
> type as well as the "I'm an ISP and I want to provide a mirror to Debian
> users so we can reduce our uplink connection a bit" type of situations.
> 
> However, I've not been able to come up with a scheme which is simple
> enough to be doable on a LAN while at the same time be usable by larger
> network providers, *and* which can't also be abused by MitM attackers.
> 
> Perhaps it's just not something we would be able to do?

https://tracker.debian.org/pkg/squid-deb-proxy sparks into my mind,
but I cant tell which of those use cases it could tackle, did not investiage 
enough for it.
(eg  for sure wont help at ISP level) 

-- 
tobi

Re: Q: Use https for {deb,security}.debian.org by default

[Andrew M.A. Cater]

On Sat, Aug 21, 2021 at 12:31:26PM +0200, Simon Richter wrote:
> Hi,
> 
> On 21.08.21 10:40, Wouter Verhelst wrote:
> 
> > I've been thinking for a while that we should bake a feature in apt
> > whereby a network administrator can indicate somehow that there is a
> > local apt mirror and that apt should use that one in preference to
> > deb.debian.org.
> 
> I've been thinking the same thing, but that would negate the remaining
> security benefit of using https, that's why I expressed a preference for
> making it visible that the connection is not encrypted and security is only
> provided by the signatures over pretending it is and then (silently?)
> allowing a proxy to intercept the connection.
> 
>Simon
> 

This is effectively what you do if you specify a non-country mirror in setup
/ specify manually anyway - and that can be preseeded / set up with ansible
or whatever scripting you want to do en masse.

For myself, I'm not convinced that forcing https everywhere will give much
benefit in reality and may impose further complexity / a greater sense of
security without much more real security but if a setting is available
to be overridden when desired, it's not a deal breaker.

All the very best as ever,

Andy Cater

Re: Q: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 21.08.21 10:40, Wouter Verhelst wrote:


I've been thinking for a while that we should bake a feature in apt
whereby a network administrator can indicate somehow that there is a
local apt mirror and that apt should use that one in preference to
deb.debian.org.


I've been thinking the same thing, but that would negate the remaining 
security benefit of using https, that's why I expressed a preference for 
making it visible that the connection is not encrypted and security is 
only provided by the signatures over pretending it is and then 
(silently?) allowing a proxy to intercept the connection.


   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Wouter Verhelst]

On Fri, Aug 20, 2021 at 07:20:22PM +, Jeremy Stanley wrote:
> Yes transparent proxies or overridden DNS lookups could be used to
> direct deb.debian.org and security.debian.org to your alternative
> location,

I've been thinking for a while that we should bake a feature in apt
whereby a network administrator can indicate somehow that there is a
local apt mirror and that apt should use that one in preference to
deb.debian.org.

This could be useful for both the "I've got a slow uplink and would like
it to not be overwhelmed at the BSP I'm hosting for my Debian friends"
type as well as the "I'm an ISP and I want to provide a mirror to Debian
users so we can reduce our uplink connection a bit" type of situations.

However, I've not been able to come up with a scheme which is simple
enough to be doable on a LAN while at the same time be usable by larger
network providers, *and* which can't also be abused by MitM attackers.

Perhaps it's just not something we would be able to do?

-- 
 w@uter.{be,co.za}
wouter@{grep.be,fosdem.org,debian.org}

Re: Q: Use https for {deb,security}.debian.org by default

[Wouter Verhelst]

On Thu, Aug 19, 2021 at 10:11:33PM +, Jeremy Stanley wrote:
> On 2021-08-19 16:37:13 -0400 (-0400), Kyle Edwards wrote:
> > On 8/19/21 3:46 PM, Simon Richter wrote:
> > > For the most part, users would configure https if they are behind a
> > > corporate firewall that disallows http, or modifies data in-flight so
> > > signature verification fails, everyone else is better off using plain
> > > http.
> > 
> > Or they might configure https on the sheer principle of not wanting to have
> > their traffic hoovered up by their ISP or anyone else who might be
> > listening.
> 
> While this does complicate it, a snooping party can still know the
> site they're connecting to via SNI happening unencrypted,

SNI is not unencrypted if you do TLS1.3...

-- 
 w@uter.{be,co.za}
wouter@{grep.be,fosdem.org,debian.org}

Re: Q: Use https for {deb,security}.debian.org by default

[Tomas Pospisek]

On 21.08.21 09:14, Philipp Kern wrote:

On 20.08.21 21:11, Russ Allbery wrote:

The way I would put it is that the security benefit of using TLS for apt
updates is primarily that it makes certain classes of attempts to mess
with the update channel more noisy and more likely to produce immediate
errors.
One thing of note is that it introduces a time dependency on the client. 
Now we seem to gravitate towards a world where you'd also fail DNS 
resolution if your time is wrong (because you cannot get at the 
DNS-over-TLS/HTTPS server), so this is probably accepted as not making 
things worse overall. I guess we could have some (somewhat insecure) 
defense in depth if we wanted to, but maybe the world just agreed that 
you need to get your clock roughly correct. ;-)


I remember seeing apt-get refusing to update packages or the index 
because of them "having timestamps in the future" or in other words 
system time being out of sync in direction of the past.


So we already have the situation that system time **must not** be off 
into the past by some delta in order to be able to do updates **at all**.


This is out of my memory so if anybody cares about this argument it 
should maybe be confirmed more thoroughly.

*t

Re: Q: Use https for {deb,security}.debian.org by default

[Hideki Yamane]

Hi all,

 Thanks for your comments!
 It seems that no big blocker to make https default for deb.debian.org
 and security.debian.org.


On Thu, 19 Aug 2021 22:38:20 +0900
Hideki Yamane  wrote:
>  Now deb.debian.org and security.debian.org provide https access
>  but created sources.list file use http for those. Is there any
>  reason to use http instead of https for them? (traffic, policy,
>  etc...) If not, how about to change it?

 Q: Make https as default situation worse?
   A: No :)
  If the clock setting is not appropriate, d-i provides NTP setting
  through installation. 

 Q: Is there any benefit?
   A: Yes, going forward with https as default is a trend, and some
  people complain about the way for accessing our repo as http.
  We can avoid such boring discussion and wrong message to users.

 Q: How about largely deploy environment like containers?
   A: They can choose http for that, since just "https as default"

 Q: Is it perfect solution?
   A: Of course not :)

 Q: Do you intend to mandate https access for apt?
   A: No, just make https "default" for deb.debian.org and security.debian.org.
  There is a choise to use other repo and http instead.



-- 
Hideki Yamane 

Re: Q: Use https for {deb,security}.debian.org by default

[Philipp Kern]

On 20.08.21 21:11, Russ Allbery wrote:

The way I would put it is that the security benefit of using TLS for apt
updates is primarily that it makes certain classes of attempts to mess
with the update channel more noisy and more likely to produce immediate
errors.
One thing of note is that it introduces a time dependency on the client. 
Now we seem to gravitate towards a world where you'd also fail DNS 
resolution if your time is wrong (because you cannot get at the 
DNS-over-TLS/HTTPS server), so this is probably accepted as not making 
things worse overall. I guess we could have some (somewhat insecure) 
defense in depth if we wanted to, but maybe the world just agreed that 
you need to get your clock roughly correct. ;-)


Kind regards
Philipp Kern

Re: Q: Use https for {deb,security}.debian.org by default

[Tim Woodall]

On Sat, 21 Aug 2021, Vincent Lefevre wrote:


On 2021-08-20 12:11:30 -0700, Russ Allbery wrote:

The most naive attempt to mess with the update channel (intercepting the
http connection and replacing a package with a malicious one) will fail
immediately with both http or https.  The primary difference in that case
with https is that the the network connection will fail (assuming no
compromise of the TLS certificate authority chain, which is possible of
course and which degrades to the http case), whereas with http you will
download the malicious package first and then apt will refuse to install

   ^^

it when the hash doesn't match.  That difference mostly doesn't matter.


But what if one doesn't install packages with apt?

I use the sources.list also to download the source with "apt source".

And what about dget?



Surely anyone as 'off piste' as that will be perfectly capable of fixing
sources.list to do what they want.

As far as 'what is the default', I really don't care. I have my own
reasons for preferring HTTP for my own 'off piste' work but I'll change
the default if necessary.

My biggest fear in all of this discussion is that after make https the
default is done, the next call will be to 'stop supporting http to force
those noob users who haven't updated their sources.list to update'

There are just two use cases I can think of where https might be
important for a 'noob' user: 1, they've got a sources.list entry that
uses basic authentication (but I don't think this is relevant to any of
the entries under discussion) and 2. They're downloading tor or
something like that in a region where it's banned (but as others have
said, relying on https to hide that is naive at best)

So, I want to register my interest in continuing to support http for
debian sources.list for all of the tools even if it's not the default
and it's not recommended.


ObFrustration: It's pretty easy, conceptually, to intercept any HTTP
traffic on the host and 'proxy' it to https off the host. It's also easy
to do that 'at the edge' (whereever you design the edge) and that way
you have one place to configure everything you care about. Instead, I
have to deal with a ridiculously complicated setup to 'peek and splice'
https to try and restrict how much information leaks to facebook et al.
And I'm well aware that 'plans are afoot' to prevent me limiting what
information leaks. And this is just a home setup. I feel for the guys
who have to deal with this in a commercial environment.

Re: Q: Use https for {deb,security}.debian.org by default

[Vincent Lefevre]

On 2021-08-20 12:11:30 -0700, Russ Allbery wrote:
> The most naive attempt to mess with the update channel (intercepting the
> http connection and replacing a package with a malicious one) will fail
> immediately with both http or https.  The primary difference in that case
> with https is that the the network connection will fail (assuming no
> compromise of the TLS certificate authority chain, which is possible of
> course and which degrades to the http case), whereas with http you will
> download the malicious package first and then apt will refuse to install
^^
> it when the hash doesn't match.  That difference mostly doesn't matter.

But what if one doesn't install packages with apt?

I use the sources.list also to download the source with "apt source".

And what about dget?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Q: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Ansgar  writes:
> On Fri, 2021-08-20 at 23:37 +0200, Simon Richter wrote:

>> I was thinking of VPS hosting for the most part, where users will run
>> apt or auto-apt inside their virtual server.

> Hosting providers or ISPs messing with their customers' traffic with
> Debian mirrors seems like something we should not support.

Agreed.  This seems like a feature, not a bug.

-- 
Russ Allbery (r...@debian.org)  

Re: Q: Use https for {deb,security}.debian.org by default

[Ansgar]

On Fri, 2021-08-20 at 23:37 +0200, Simon Richter wrote:
> I was thinking of VPS hosting for the most part, where users will run
> apt or auto-apt inside their virtual server.

Hosting providers or ISPs messing with their customers' traffic with
Debian mirrors seems like something we should not support.

Ansgar

Re: Q: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 8/20/21 9:04 PM, Russ Allbery wrote:


One of the things that confuses me about this user story is why are your
containers doing non-trivial amounts of apt traffic at runtime?  Generally
the whole point of a container is that you only do this during container
build time.  I'm not sure I understand how you have gotten into a
situation where you have containers that you can't modify but that are
self-modifying (by updating apt packages) at runtime.


I was thinking of VPS hosting for the most part, where users will run 
apt or auto-apt inside their virtual server.


Building containers seems to be mostly harmless, at least no mirror 
operator has complained so far when I do a few hundred rebuilds for 
testing and download 500 MB of packages every time.


   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Ansgar]

On Fri, 2021-08-20 at 12:11 -0700, Russ Allbery wrote:
> The way I would put it is that the security benefit of using TLS for apt
> updates is primarily that it makes certain classes of attempts to mess
> with the update channel more noisy and more likely to produce immediate
> errors.

APT is not the only way to download packages: often enough users (and
developers) will ignore apt, download packages manually for various
reasons, *not* do the integrity checks apt does and install them.

Using https:// URLs for mirrors wherever possible makes this a bit less
bad.

Ansgar

Re: Q: Use https for {deb,security}.debian.org by default

[Johannes Schauer Marin Rodrigues]

Quoting Jeremy Stanley (2021-08-20 18:34:22)
> > If so, I think the next step would be to open a bug with a summary of this
> > discussion.  I'm happy to do that, but I'm not sure what package owns this
> > configuration.
> It's not owned directly by a particular package, I think D-I and various
> bootstrapping tools independently write it at installation, so the "fixes"
> for this are likely to be in a variety of places.

As the author of one of these "bootstrapping tools" I would appreciate if we
had the information about which distro and release would use which mirror
address stored somewhere in a machine readable format. The switch of the
security mirror line for bullseye or later would've been much easier if we
wouldn't have to fix all tools hardcoding it...

Maybe distro-info-data could carry such information?

signature.asc
Description: signature

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-20 12:04:02 -0700 (-0700), Russ Allbery wrote:
> Simon Richter  writes:
> 
> > I support that idea in principle, but one of our user stories is
> > "I have a datacenter with a few thousand containers in it, so I
> > want to redirect accesses to the local mirror to reduce external
> > network traffic."
> 
> Just checking that I understand.  You have several thousand
> containers that you're running in your data center but cannot
> modify and whose network access specifically to Debian apt mirrors
> you want to intercept and redirect, and you're relying on them
> using http instead of https in order to be able to do this?
[...]

Probably the more general case here is that you have a lot of
systems (I agree "containers" is a bit preposterous when we're
talking about package installs/updates) for which you want to
provide some sort of cache/proxy/mirror and for whatever reason be
it convenience or resource constraints want to serve those over
HTTP. But the answer here is that changing the default sources.list
entries written by debian-installer or baked into pre-built "cloud
images" pretty much always comes into play anyway. You're going to
be modifying the systems' sources.list entries already via
configuration management or some agent like cloud-init to be able to
use that cache/proxy/mirror, so listing the correct transport scheme
in your modified entries isn't additional effort.

Yes transparent proxies or overridden DNS lookups could be used to
direct deb.debian.org and security.debian.org to your alternative
location, but in my experience (working at colocation/hosting
companies as well as cloud service providers) this is rarely done
and generally considered unhygienic. Transparent "web accelerators"
used to be popular in such environments, but the modern trend to
switch most communications to HTTPS has rendered them essentially
useless since years.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Paul Gevers  writes:

> I was told and I relayed early in this thread [1] that https gives you
> some (delayed) protection against man-in-the-middle attacks serving you
> old data.

Yes, it gives you some protection.  Jeremy is more cynical about the
utility of that protection than I am, although it's certainly arguable how
much it's likely to matter in practice.

The way I would put it is that the security benefit of using TLS for apt
updates is primarily that it makes certain classes of attempts to mess
with the update channel more noisy and more likely to produce immediate
errors.

The most naive attempt to mess with the update channel (intercepting the
http connection and replacing a package with a malicious one) will fail
immediately with both http or https.  The primary difference in that case
with https is that the the network connection will fail (assuming no
compromise of the TLS certificate authority chain, which is possible of
course and which degrades to the http case), whereas with http you will
download the malicious package first and then apt will refuse to install
it when the hash doesn't match.  That difference mostly doesn't matter.

-- 
Russ Allbery (r...@debian.org)  

Re: Q: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Simon Richter  writes:

> I support that idea in principle, but one of our user stories is "I have
> a datacenter with a few thousand containers in it, so I want to redirect
> accesses to the local mirror to reduce external network traffic."

Just checking that I understand.  You have several thousand containers
that you're running in your data center but cannot modify and whose
network access specifically to Debian apt mirrors you want to intercept
and redirect, and you're relying on them using http instead of https in
order to be able to do this?

One of the things that confuses me about this user story is why are your
containers doing non-trivial amounts of apt traffic at runtime?  Generally
the whole point of a container is that you only do this during container
build time.  I'm not sure I understand how you have gotten into a
situation where you have containers that you can't modify but that are
self-modifying (by updating apt packages) at runtime.

-- 
Russ Allbery (r...@debian.org)  

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-20 20:52:43 +0200 (+0200), Paul Gevers wrote:
[...]
> I was told and I relayed early in this thread [1] that https gives you
> some (delayed) protection against man-in-the-middle attacks serving you
> old data. Does everybody agree that this is either not prevented or not
> giving you more security or is an extremely unlikely security threat?
> Because when I thought about it, it did make sense, but I'm not somebody
> that thinks about security for a profession, nor do I claim to be any
> expert at all on the topic.
> 
> I'm not saying that explaining this is worth more than the
> https-as-default-for-the-noob reasoning, I just want to know what you
> think about the argument.
[...]

It shrinks the window, but there are already safeguards preventing
use of stale indices based on coarse timestamping. At best a MitM
could quietly block you from downloading security updates until the
old indices they're injecting expire, but they can also more noisily
prevent you from downloading security updates for far longer,
regardless of whether you use HTTPS as a transport.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Paul Gevers]

Hi,

On 20-08-2021 17:48, Russ Allbery wrote:
> It sounds like we have a general consensus in this thread that, while
> changing our default to HTTPS probably won't make anything more secure in
> practice, we should still do it?

I was told and I relayed early in this thread [1] that https gives you
some (delayed) protection against man-in-the-middle attacks serving you
old data. Does everybody agree that this is either not prevented or not
giving you more security or is an extremely unlikely security threat?
Because when I thought about it, it did make sense, but I'm not somebody
that thinks about security for a profession, nor do I claim to be any
expert at all on the topic.

I'm not saying that explaining this is worth more than the
https-as-default-for-the-noob reasoning, I just want to know what you
think about the argument.

Paul

[1] https://lists.debian.org/debian-devel/2021/08/msg00277.html



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/20/21 2:37 PM, Simon Richter wrote:


This is a use case where HTTPS does hurt, and where I can't think of 
any good mitigation strategies that wouldn't be worse from a security 
PoV than the status quo.


Such situations are the exception rather than the norm. If https is 
detrimental to their setup, they can choose to opt out of it.


For everyone else, I think https should be the default.

Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,


"Use HTTPS everywhere that supports it" is simple and actionable advice
for the average person that will make them more secure.



There are
applications and sites where HTTPS doesn't really help, but other than
some unusual performance edge cases that are pretty rare in practice, it
doesn't hurt.


I support that idea in principle, but one of our user stories is "I have 
a datacenter with a few thousand containers in it, so I want to redirect 
accesses to the local mirror to reduce external network traffic."


This is a use case where HTTPS does hurt, and where I can't think of any 
good mitigation strategies that wouldn't be worse from a security PoV 
than the status quo.


   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-20 08:48:11 -0700 (-0700), Russ Allbery wrote:
[...]
> It sounds like we have a general consensus in this thread that,
> while changing our default to HTTPS probably won't make anything
> more secure in practice, we should still do it?

Coincident with any default change, it would probably be a good idea
to make it clear this decision is not based on inherent insecurity
of Debian's package distribution channels, but rather primarily for
alignment with prevailing industry guidance on the general avoidance
of bare HTTP in order to reduce confusion for users. Relying on
local package proxies/caches or mirrors which lack HTTPS (due
primarily to the complexity and inconvenience of buying certs or
setting up something like certbot) is likely to continue, and is
still similarly safe, so it would be unfortunate to inadvertently
give the impression it's not.

The "https" scheme entry under URI SPECIFICATION in sources.list(5)
already does a fairly good job of explaining that it doesn't
necessarily hide what's being downloaded and from where, indicating
it's mainly helpful for protecting credentials where basic auth is
used (as inferred from the "http" section before it). The "http"
section also provides a concise explanation of the existing
integrity measures independent of transport choices, and refers
readers to apt-secure(8) for more information. So long as this
messaging is retained and reinforced, switching the default
sources.list entries does seem like a perfectly pragmatic choice.

> If so, I think the next step would be to open a bug with a summary
> of this discussion.  I'm happy to do that, but I'm not sure what
> package owns this configuration.

It's not owned directly by a particular package, I think D-I and
various bootstrapping tools independently write it at installation,
so the "fixes" for this are likely to be in a variety of places.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Jeremy Stanley  writes:

> Yes, this is a much nicer way of rephrasing it, but basically still what
> I said. Turning on HTTPS by default wouldn't be addressing any
> particular user risk, it would simply keep everyone from having to
> discuss and explain it ad nauseum. Much like replacing older hash
> functions for non-security-related checksumming: the potential for
> accidental collisions may not outweigh the engineering effort required
> to update applications or protocols, but the time developers will spend
> having to explain why they're using MD5 or SHA-1 hashes can be orders of
> magnitude greater still.

Yes, this sounds right to me.

It sounds like we have a general consensus in this thread that, while
changing our default to HTTPS probably won't make anything more secure in
practice, we should still do it?

If so, I think the next step would be to open a bug with a summary of this
discussion.  I'm happy to do that, but I'm not sure what package owns this
configuration.

-- 
Russ Allbery (r...@debian.org)  

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-20 07:56:33 -0700 (-0700), Russ Allbery wrote:
[...]
> Do you think using HTTPS makes security worse?
[...]

No, obviously not, except insofar that instilling a false sense of
security can be harmful in the long term because it excuses people
from thinking about the actual problems they're hoping to solve.

> Personally, I think we should switch our default to HTTPS not
> because we have a specific security flaw in mind against which
> HTTPS provides some protection but because it's consistent with
> the general message that a lot of us (including, for example, the
> EFF and the IETF) are trying to send to average users who don't
> have the expertise to analyze any of this: use TLS by default
> wherever you can.

Yes, this is a much nicer way of rephrasing it, but basically still
what I said. Turning on HTTPS by default wouldn't be addressing any
particular user risk, it would simply keep everyone from having to
discuss and explain it ad nauseum. Much like replacing older hash
functions for non-security-related checksumming: the potential for
accidental collisions may not outweigh the engineering effort
required to update applications or protocols, but the time
developers will spend having to explain why they're using MD5 or
SHA-1 hashes can be orders of magnitude greater still.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/20/21 10:56 AM, Russ Allbery wrote:

Do you think using HTTPS makes security worse?

No idea whether I qualify as a "security expert" but as someone who has
spent a fair amount of time working in security, my concern is making
advice simple enough for people to follow.  Complicated, conditional, or
inconsistent advice means you lose people who decide this is all too hard
to understand and just do nothing.

"Use HTTPS everywhere that supports it" is simple and actionable advice
for the average person that will make them more secure.  There are
applications and sites where HTTPS doesn't really help, but other than
some unusual performance edge cases that are pretty rare in practice, it
doesn't hurt.  It's not magic fairy dust, but it does raise the bar
against a set of attacks, provides some additional privacy against casual
non-targeted snooping, and is a better default than not using TLS.

Personally, I think we should switch our default to HTTPS not because we
have a specific security flaw in mind against which HTTPS provides some
protection but because it's consistent with the general message that a lot
of us (including, for example, the EFF and the IETF) are trying to send to
average users who don't have the expertise to analyze any of this: use TLS
by default wherever you can.  It's not a panacea, but ubiquitous, default
use of TLS helps both your security and your privacy compared to either
the previous default of no TLS or spending a bunch of mental energy
picking and choosing.


I agree with all of this. While HTTPS might not be sufficient, it is 
still necessary. If the last decade of leaks, breaches, and scandals 
have taught us anything, it's that strong encryption should be the 
default, not an afterthought. We should be moving towards more 
encryption, not less.


Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Jeremy Stanley  writes:

> I agree with all of the above, my point was that the current state of
> HTTPS doesn't especially improve integrity for Debian package management
> over the signed indices and checksums we already rely on, and trying to
> use HTTPS for privacy/secrecy (which isn't really what it was designed
> for) is still and perhaps even increasingly misguided. Of course lots of
> people will continue to expect magic HTTPS fairy dust to protect them
> and ward off evil, but the only legitimate reason I can see for Debian
> changing the default protocol for sources.list entries is to avoid
> having to pointlessly debate the minimal benefits of HTTPS with people
> who drink whatever cool-aid they're told by security "experts" (HTTP
> bad, HTTPS good, drink up!).

Do you think using HTTPS makes security worse?

No idea whether I qualify as a "security expert" but as someone who has
spent a fair amount of time working in security, my concern is making
advice simple enough for people to follow.  Complicated, conditional, or
inconsistent advice means you lose people who decide this is all too hard
to understand and just do nothing.

"Use HTTPS everywhere that supports it" is simple and actionable advice
for the average person that will make them more secure.  There are
applications and sites where HTTPS doesn't really help, but other than
some unusual performance edge cases that are pretty rare in practice, it
doesn't hurt.  It's not magic fairy dust, but it does raise the bar
against a set of attacks, provides some additional privacy against casual
non-targeted snooping, and is a better default than not using TLS.

Personally, I think we should switch our default to HTTPS not because we
have a specific security flaw in mind against which HTTPS provides some
protection but because it's consistent with the general message that a lot
of us (including, for example, the EFF and the IETF) are trying to send to
average users who don't have the expertise to analyze any of this: use TLS
by default wherever you can.  It's not a panacea, but ubiquitous, default
use of TLS helps both your security and your privacy compared to either
the previous default of no TLS or spending a bunch of mental energy
picking and choosing.

-- 
Russ Allbery (r...@debian.org)  

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-20 11:36:41 +0200 (+0200), Bjørn Mork wrote:
> Jeremy Stanley  writes:
> 
> > While this does complicate it, a snooping party can still know the
> > site they're connecting to via SNI happening unencrypted,
> 
> I believe this can be fixed with TLS 1.3?
> 
> > and packet sizes/pacing likely give away which pages or files are
> > being retrieved based on their length.
> 
> Yes, probably looking into territory where you'd not want to directly
> access any public service at all here..
> 
> > And that's not even getting into
> > how "trusted" certificate authorities give away certificates for any
> > hostname if your MitM knows the right people,
> 
> Debian is among the few who publish TLSA records (DANE).  Which is still
> pretty useless for normal web srvices since the major browser vendors
> refuse to support it.  But TLSA validation could easily be implemented
> in apt-transport-https. Maybe it is?  That would prevent this problem.
> 
> > and CDNs are now in
> > the business of snooping on everyone's traffic for sites where they
> > handle SSL/TLS termination. HTTPS as deployed on the open Internet
> > is a sip of security with several gulps of theater.
> 
> Not much to do if you don't trust your own servers, whether they are CDN
> frontends or whatever.

I agree with all of the above, my point was that the current state
of HTTPS doesn't especially improve integrity for Debian package
management over the signed indices and checksums we already rely on,
and trying to use HTTPS for privacy/secrecy (which isn't really what
it was designed for) is still and perhaps even increasingly
misguided. Of course lots of people will continue to expect magic
HTTPS fairy dust to protect them and ward off evil, but the only
legitimate reason I can see for Debian changing the default protocol
for sources.list entries is to avoid having to pointlessly debate
the minimal benefits of HTTPS with people who drink whatever
cool-aid they're told by security "experts" (HTTP bad, HTTPS good,
drink up!).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Bjørn Mork]

Jeremy Stanley  writes:

> While this does complicate it, a snooping party can still know the
> site they're connecting to via SNI happening unencrypted,

I believe this can be fixed with TLS 1.3?

> and packet sizes/pacing likely give away which pages or files are
> being retrieved based on their length.

Yes, probably looking into territory where you'd not want to directly
access any public service at all here..

> And that's not even getting into
> how "trusted" certificate authorities give away certificates for any
> hostname if your MitM knows the right people,

Debian is among the few who publish TLSA records (DANE).  Which is still
pretty useless for normal web srvices since the major browser vendors
refuse to support it.  But TLSA validation could easily be implemented
in apt-transport-https. Maybe it is?  That would prevent this problem.

> and CDNs are now in
> the business of snooping on everyone's traffic for sites where they
> handle SSL/TLS termination. HTTPS as deployed on the open Internet
> is a sip of security with several gulps of theater.

Not much to do if you don't trust your own servers, whether they are CDN
frontends or whatever.



Bjørn

Re: Q: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-08-19 16:37:13 -0400 (-0400), Kyle Edwards wrote:
> On 8/19/21 3:46 PM, Simon Richter wrote:
> > For the most part, users would configure https if they are behind a
> > corporate firewall that disallows http, or modifies data in-flight so
> > signature verification fails, everyone else is better off using plain
> > http.
> 
> Or they might configure https on the sheer principle of not wanting to have
> their traffic hoovered up by their ISP or anyone else who might be
> listening.

While this does complicate it, a snooping party can still know the
site they're connecting to via SNI happening unencrypted, and packet
sizes/pacing likely give away which pages or files are being
retrieved based on their length. And that's not even getting into
how "trusted" certificate authorities give away certificates for any
hostname if your MitM knows the right people, and CDNs are now in
the business of snooping on everyone's traffic for sites where they
handle SSL/TLS termination. HTTPS as deployed on the open Internet
is a sip of security with several gulps of theater.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/19/21 3:46 PM, Simon Richter wrote:
For the most part, users would configure https if they are behind a 
corporate firewall that disallows http, or modifies data in-flight so 
signature verification fails, everyone else is better off using plain 
http.


Or they might configure https on the sheer principle of not wanting to 
have their traffic hoovered up by their ISP or anyone else who might be 
listening.


Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Philipp Kern]

On 19.08.21 21:48, Paul Gevers wrote:

On 19-08-2021 21:46, Simon Richter wrote:

For the most part, users would configure https if they are behind a
corporate firewall that disallows http, or modifies data in-flight so
signature verification fails, everyone else is better off using plain http.

Except for the security archive, where https can prevent a
man-in-the-middle from serving you outdated information and thus deprive
you from updates.


For a week until Valid-Until expires. Note that the denial of service 
equally works for HTTPS, it's just more noisy.


Kind regards
Philipp Kern

Re: Q: Use https for {deb,security}.debian.org by default

[Paul Gevers]

Hi,

On 19-08-2021 21:46, Simon Richter wrote:
> For the most part, users would configure https if they are behind a
> corporate firewall that disallows http, or modifies data in-flight so
> signature verification fails, everyone else is better off using plain http.

Except for the security archive, where https can prevent a
man-in-the-middle from serving you outdated information and thus deprive
you from updates.

Paul



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 8/19/21 3:38 PM, Hideki Yamane wrote:


  Now deb.debian.org and security.debian.org provide https access
  but created sources.list file use http for those. Is there any
  reason to use http instead of https for them? (traffic, policy,
  etc...) If not, how about to change it?


There is little benefit to do so, it just increases processing overhead 
and breaks caching proxies, most importantly transparent proxies in 
large hosting companies and large container deployments.


For the most part, users would configure https if they are behind a 
corporate firewall that disallows http, or modifies data in-flight so 
signature verification fails, everyone else is better off using plain http.


   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Re: Q: Use https for {deb,security}.debian.org by default

[Tim Woodall]

Is there any way to estimate what proportion of clients are behind a
proxy? security.debian.org in particular could possibly see a lot more
traffic when there are things like kernel updates.
(Clearly it's not possible to determine how many clients a caching proxy
might be serving)

Are there any plans/thoughts to drop http access?



On Thu, 19 Aug 2021, Hideki Yamane wrote:


Hi,

Now deb.debian.org and security.debian.org provide https access
but created sources.list file use http for those. Is there any
reason to use http instead of https for them? (traffic, policy,
etc...) If not, how about to change it?

Q: Use https for {deb,security}.debian.org by default

[Hideki Yamane]

Hi,

 Now deb.debian.org and security.debian.org provide https access
 but created sources.list file use http for those. Is there any
 reason to use http instead of https for them? (traffic, policy,
 etc...) If not, how about to change it?


-- 
Hideki Yamane