Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Sun, Sep 12, 2021 at 03:10:27AM +, Paul Wise wrote:
> On Fri, Sep 10, 2021 at 6:03 PM David Kalnischkies wrote:
> > Because this thread started with the idea to switch the default of d-i
> > and co to another URI. If you target only apt then you still need
> > a solution for d-i and a way to convert whatever d-i had into what apt
> > gets in the end (of the installation).
> 
> ISTR the future of creating new Debian installations is to move from
> debootstrap to dpkg/apt. As an interim step, debootstrap could move
> from doing its own downloads to passing the appropriate
> APT_CONFIG/DPKG_ROOT/etc to `apt download`.
> 
> https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap

The spec deals with the installation of the essential set.
APT isn't essential – it is 'only' one of the first packages installed
after the bootstrap is done, usually at least.

Moving {,c}debootstrap to use apt means you increase the system
requirements from "can execute debootstrap" all the way up to "is
a fully bootstrapped Debian-based system". At which point you could
just use mmdebstrap instead of debootstrap and be done.

I am not involved with d-i to know if they would plan such a move, but
I have at least never heard of it and it seems outside the linked spec.
You might have confused this with the pipe-dream of obsoleting
mmdebstrap at some far away in the future point by folding it into apt
directly. The spec is one (of the many) pre-requirements for that.


Even if we do, that would move the goal post only slightly as you still
have the problem that the conf used to create the system might very well
not be the conf that can be used by the created system (as a trivial
example some old apt versions do not support https). That doesn't really
change regardless of using anna, debootstrap, apt or whatever else.


Best regards

David Kalnischkies

P.S.: Having apt be involved in its own bootstrap reminds me of that
time when I saved myself from drowning in a swamp by pulling on my hair…
https://en.wikipedia.org/wiki/Baron_Munchausen#Fictional_character


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Holger Levsen]

On Sun, Sep 12, 2021 at 03:10:27AM +, Paul Wise wrote:
> ISTR the future of creating new Debian installations is to move from
> debootstrap to dpkg/apt. As an interim step, debootstrap could [...]

I've switched all my occurances of using debootstrap to mmdebstrap and
am a very happy user of it.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Paul Wise]

On Fri, Sep 10, 2021 at 6:03 PM David Kalnischkies wrote:

> Because this thread started with the idea to switch the default of d-i
> and co to another URI. If you target only apt then you still need
> a solution for d-i and a way to convert whatever d-i had into what apt
> gets in the end (of the installation).

ISTR the future of creating new Debian installations is to move from
debootstrap to dpkg/apt. As an interim step, debootstrap could move
from doing its own downloads to passing the appropriate
APT_CONFIG/DPKG_ROOT/etc to `apt download`.

https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Fri, Sep 10, 2021 at 08:02:42PM +0200, David Kalnischkies wrote:

On Fri, Sep 10, 2021 at 11:08:38AM -0400, Michael Stone wrote:

On Fri, Sep 10, 2021 at 04:33:42PM +0200, David Kalnischkies wrote:
> On Thu, Sep 09, 2021 at 08:53:21AM -0400, Michael Stone wrote:
> > The only thing I could see that would be a net gain would be to generalizes
> > sources.list more. Instead of having a user select a specific protocol and
> > path, allow the user to just select high-level objects. Make this a new
> > pseudo-protocol for backward compatibility, and introduce something like
> >   deb apt:// suite component[s]
> > so the default could be something like
> >   deb apt:// bullseye main
> >   deb apt:// bullseye/updates main
> > then the actual protocols, servers, and paths could be managed by various
> > plugins and overridden by configuration directives in apt.conf.d. For
>
> In this scheme the Debian bullseye main repo has the same 'URI' as the
> Darts bullseye main repo. So, you would need to at least include an
> additional unique identifier of the likes of Debian and Darts, but
> who is to assign those URNs?
> (Currently we are piggybacking on the domain name system for this)

I have no idea what darts is, so I don't have an answer. :)


"Darts" was just a play on "bullseye". It is not hard to imagine
a repository which has the same suite and component(s) but is not
Debian itself. As a pseudo-random [= its in an other topic here] real
example you can take Wine (https://dl.winehq.org/wine-builds/debian/).
So to what is "deb apt:// bullseye main" referring? Debian or Wine?

And to pre-empt the most common response: As an apt dev I can assure you
that we won't accept a solution involving "I am on Debian, so it means
Debian" as that is impossible to correctly guess programmatically (for
example on derivatives using a small overlay repo).


I'd considered adding a scope to the model, e.g., apt://debian.org but 
removed it for simplicity. If that was a desired feature then there'd 
either have to be some sort of well-known path or such a distribution 
would need to provide a policy plugin for that scope. Alternatively, 
they could just use the existing http/https/whatever syntax in 
sources.list for their overlay if they didn't want to bother. Same for 
third party repos. 


> > their thing, and a plugin like auto-apt-proxy can override defaults to do
> > something more complicated, using more policy-friendly .d configurations
> > rather than figuring out a way to rewrite some other package's configuration
> > file.
>
> JFTR: auto-apt-proxy has nothing to do with sources. It is true that
> apt-cacher-ng (and perhaps others) have a mode of operation in which you
> prefix the URI of your (in that case caching) proxy to the URI of the
> actual repo, but that isn't how a proxy usually operates and/or is
> configured.

I have no idea what you're saying here.


And I have no idea if you know what you are talking about.

auto-apt-proxy already uses an interface apt provides to configure the
proxy at runtime. It isn't in the business of modifying sources.list nor
has it any interest in that. So you using it as an example for a plugin
who could use your proposed scheme to modify the sources at runtime
makes no sense.


The concern I was responding to was that switching to https breaks the 
case of using auto-apt-proxy to cache the transaction. Just turning the 
proxy on and off isn't sufficient if the default sources.list uses https 
instead of http--you'd currently have to both turn the proxy on *and* 
change sources.list from http to https. Hence my musings on whether it's 
possible/desirable to separate the configuration of what to use as a 
transport from the configuration of what repository is desired.


More generally, if we're talking about changing the default way that 
people interact with debian it just seems like a good time to ponder 
whether specifying sources the same way we did in 1998 makes sense given 
how many changes there have been to expectations about how we use
internet resources. Maybe the answer is yes, and I'm not arguing that 
there has to be a change or that what I threw out as a possibility is 
the right answer, but it does seem worth considering.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Fri, Sep 10, 2021 at 11:08:38AM -0400, Michael Stone wrote:
> On Fri, Sep 10, 2021 at 04:33:42PM +0200, David Kalnischkies wrote:
> > On Thu, Sep 09, 2021 at 08:53:21AM -0400, Michael Stone wrote:
> > > The only thing I could see that would be a net gain would be to 
> > > generalizes
> > > sources.list more. Instead of having a user select a specific protocol and
> > > path, allow the user to just select high-level objects. Make this a new
> > > pseudo-protocol for backward compatibility, and introduce something like
> > >   deb apt:// suite component[s]
> > > so the default could be something like
> > >   deb apt:// bullseye main
> > >   deb apt:// bullseye/updates main
> > > then the actual protocols, servers, and paths could be managed by various
> > > plugins and overridden by configuration directives in apt.conf.d. For
> > 
> > In this scheme the Debian bullseye main repo has the same 'URI' as the
> > Darts bullseye main repo. So, you would need to at least include an
> > additional unique identifier of the likes of Debian and Darts, but
> > who is to assign those URNs?
> > (Currently we are piggybacking on the domain name system for this)
> 
> I have no idea what darts is, so I don't have an answer. :)

"Darts" was just a play on "bullseye". It is not hard to imagine
a repository which has the same suite and component(s) but is not
Debian itself. As a pseudo-random [= its in an other topic here] real
example you can take Wine (https://dl.winehq.org/wine-builds/debian/).
So to what is "deb apt:// bullseye main" referring? Debian or Wine?

And to pre-empt the most common response: As an apt dev I can assure you
that we won't accept a solution involving "I am on Debian, so it means
Debian" as that is impossible to correctly guess programmatically (for
example on derivatives using a small overlay repo).


> > Also, but just as an aside, whatever clever system you think of apt
> > could be using, you still need a rather simple system for the likes of
> > tools which come before apt like the installers/bootstrappers as they
> > are not (all) using apt, especially not in the very early stages, and
> > a mapping between them.
> 
> I'm not sure why you think I need that? The goal of my musings is to

Because this thread started with the idea to switch the default of d-i
and co to another URI. If you target only apt then you still need
a solution for d-i and a way to convert whatever d-i had into what apt
gets in the end (of the installation).

The configuration option which only works with apt tools already
exists in the form of the mirror method…


> > > their thing, and a plugin like auto-apt-proxy can override defaults to do
> > > something more complicated, using more policy-friendly .d configurations
> > > rather than figuring out a way to rewrite some other package's 
> > > configuration
> > > file.
> > 
> > JFTR: auto-apt-proxy has nothing to do with sources. It is true that
> > apt-cacher-ng (and perhaps others) have a mode of operation in which you
> > prefix the URI of your (in that case caching) proxy to the URI of the
> > actual repo, but that isn't how a proxy usually operates and/or is
> > configured.
> 
> I have no idea what you're saying here.

And I have no idea if you know what you are talking about.

auto-apt-proxy already uses an interface apt provides to configure the
proxy at runtime. It isn't in the business of modifying sources.list nor
has it any interest in that. So you using it as an example for a plugin
who could use your proposed scheme to modify the sources at runtime
makes no sense.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Fri, Sep 10, 2021 at 04:33:42PM +0200, David Kalnischkies wrote:

On Thu, Sep 09, 2021 at 08:53:21AM -0400, Michael Stone wrote:

The only thing I could see that would be a net gain would be to generalizes
sources.list more. Instead of having a user select a specific protocol and
path, allow the user to just select high-level objects. Make this a new
pseudo-protocol for backward compatibility, and introduce something like
  deb apt:// suite component[s]
so the default could be something like
  deb apt:// bullseye main
  deb apt:// bullseye/updates main
then the actual protocols, servers, and paths could be managed by various
plugins and overridden by configuration directives in apt.conf.d. For


In this scheme the Debian bullseye main repo has the same 'URI' as the
Darts bullseye main repo. So, you would need to at least include an
additional unique identifier of the likes of Debian and Darts, but
who is to assign those URNs?
(Currently we are piggybacking on the domain name system for this)


I have no idea what darts is, so I don't have an answer. :)


Also, but just as an aside, whatever clever system you think of apt
could be using, you still need a rather simple system for the likes of
tools which come before apt like the installers/bootstrappers as they
are not (all) using apt, especially not in the very early stages, and
a mapping between them.


I'm not sure why you think I need that? The goal of my musings is to 
separate the definition of what set of debian packages to use from the 
decision of how to get those packages *when using apt*, so that there 
are fewer things to consider in the common case, and to allow new 
capabilities in uncommon cases. If someone's using some other tool, why 
would anyone assume that the same configuration syntax would work? Just 
use the same configuration file you use now, and ignore all of this. If 
you want configurations to match across tools, then ignore all of this 
again and keep the same sources.list syntax you're already using that 
presumably does what you want.



If someone wants to use tor by default but fall back to https if it's
unreachable, they can do that. If someone wants to use a local proxy via
http but https if they're not on their local network, they can do that. New
installations could default to https, existing installations can keep doing


You can do most of the fallback part with the mirror method backed by
a local file. It is of no concern to apt how that file comes to be, so
you could create it out of a massive amount of options or simply by
hand. I do think if the creation is tool-based it shouldn't be apt
as I envision far too many unique snowflakes for a one-size-fits-all
approach.


The intent would be to make it easier to plug other tools into apt,
not to have the apt codebase do everything.


their thing, and a plugin like auto-apt-proxy can override defaults to do
something more complicated, using more policy-friendly .d configurations
rather than figuring out a way to rewrite some other package's configuration
file.


JFTR: auto-apt-proxy has nothing to do with sources. It is true that
apt-cacher-ng (and perhaps others) have a mode of operation in which you
prefix the URI of your (in that case caching) proxy to the URI of the
actual repo, but that isn't how a proxy usually operates and/or is
configured.


I have no idea what you're saying here. 

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 10.09.21 01:46, Paul Wise wrote:


Another important argument is that it creates a dependency on
third-party commercial CDNs, and their *continued* sponsorship.



This dependency on external providers is unavoidable, Debian
definitely cannot afford to run our own CDN at the scale needed to
support our existing userbase. For example the security mirrors
struggled with Linux kernel security updates, so security.d.o switched
to a commercial CDN. Also, we are dependent on continued sponsorship
for all of our infrastructure, paying for all of our hosting is likely
not feasible.


Yes -- and mirrors have traditionally been provided by third parties. 
What is new about the CDNs is that there are rather few of those, and 
this centralization aspect is what worries me.



Personally I'd like to see a larger variety of Debian delivery
mechanisms; copy Debian/snapshot to archive.org, create a multi-distro
FLOSS CDN, bring back httpredir, DebTorrent and apt-p2p, add an i2p
mirror, use IPFS and content oriented networking etc. Michael Stone's
apt://debian idea seems like a good way to move in that direction
while adding protocol agility.


Yes, absolutely -- and we especially want to have a better solution for 
containers, because my expectation is that a large fraction of our 
traffic is just people not bothering to set up caching.


   Simon

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Thu, Sep 09, 2021 at 08:53:21AM -0400, Michael Stone wrote:
> The only thing I could see that would be a net gain would be to generalizes
> sources.list more. Instead of having a user select a specific protocol and
> path, allow the user to just select high-level objects. Make this a new
> pseudo-protocol for backward compatibility, and introduce something like
>   deb apt:// suite component[s]
> so the default could be something like
>   deb apt:// bullseye main
>   deb apt:// bullseye/updates main
> then the actual protocols, servers, and paths could be managed by various
> plugins and overridden by configuration directives in apt.conf.d. For

In this scheme the Debian bullseye main repo has the same 'URI' as the
Darts bullseye main repo. So, you would need to at least include an
additional unique identifier of the likes of Debian and Darts, but
who is to assign those URNs?
(Currently we are piggybacking on the domain name system for this)

Also, but just as an aside, whatever clever system you think of apt
could be using, you still need a rather simple system for the likes of
tools which come before apt like the installers/bootstrappers as they
are not (all) using apt, especially not in the very early stages, and
a mapping between them.


> If someone wants to use tor by default but fall back to https if it's
> unreachable, they can do that. If someone wants to use a local proxy via
> http but https if they're not on their local network, they can do that. New
> installations could default to https, existing installations can keep doing

You can do most of the fallback part with the mirror method backed by
a local file. It is of no concern to apt how that file comes to be, so
you could create it out of a massive amount of options or simply by
hand. I do think if the creation is tool-based it shouldn't be apt
as I envision far too many unique snowflakes for a one-size-fits-all
approach.

(The Tor to https fallback can be done already if we talk onion services
 to others. You can't fall out of Tor – or redirect into it – through as
 that would allow bad actors to discover who you are/that you have an
 operational tor client installed. proxy configuration you can already
 change programmatically on the fly – a job auto-apt-proxy implements –,
 changing the mirror file with a hook from your network manager would
 be equally easy.)


> their thing, and a plugin like auto-apt-proxy can override defaults to do
> something more complicated, using more policy-friendly .d configurations
> rather than figuring out a way to rewrite some other package's configuration
> file.

JFTR: auto-apt-proxy has nothing to do with sources. It is true that
apt-cacher-ng (and perhaps others) have a mode of operation in which you
prefix the URI of your (in that case caching) proxy to the URI of the
actual repo, but that isn't how a proxy usually operates and/or is
configured.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Fri, Sep 10, 2021 at 09:33:56AM +0200, Helmut Grohne wrote:

Laptops of end-user systems are the target, but also developers. When
people gather at a place (conference, hackspace, private meetup, etc.)
downloading of .debs should just work quickly by default. Many such
sites could easily provide a local cache and a number even do. BSPs tend
to have a blackboard with information including the local mirror to use.
Seriously, how many people change their mirror when they go to a BSP? If
we installed auto-apt-proxy by default, much of the local caching would
just work.


I think you'd get a lot of pushback on installing auto-apt-proxy by 
default. If that's a proposal, make it seperately and not in this 
thread. 


The thing we seem to be disagreeing on is what is more important? https
by default or quick and efficient downloads? Some may think that our
CDN can handle the load just fine and the effects of caching are
peanuts. I can attest that those peanuts are 4TB/year (equivalent to 8
days waiting for downloads) for me.



I see that we've given up on a global network of independently-managed
mirrors and that CDNs are way easier at this time. While initially CDNs
had bad reliability issues, those have completely vanished in my
experience. On the other hand, local caching still outperforms CDNs by a
factor of 10 or so. I'd love to make it the default.


I use a cache out of habit and to be a good netizen, but my internet 
connection is fast enough these days that it's basically a noop at best 
and a slight slowdown at worst. I had to move the cache from slow/cheap 
spinning disk to reasonably fast SSD to get to that point. If you're 
downloading the same stuff over and over (e.g., for testing or somesuch) 
it can be a big win, especially for VMs on a virtualized network 
connection, or if you're managing a large infrastructure, but 
for normal use with a couple of instances? It's just not worth the 
trouble. 

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Fri, Sep 10, 2021 at 12:00:57PM +0200, Timo Röhling wrote:

* Michael Stone  [2021-09-08 19:25]:
I think the issue isn't certificate validation, it's that https 
proxy requests are made via CONNECT rather than GET. You could 
theoretically rewrite the proxy mechanism to MITM the CONNECT, but 
that wouldn't be a drop-in replacement. I suppose you could instead 
add an apt option to pass the https request to the proxy via GET 
instead of using CONNECT, but I think that also won't necessarily 
work on an existing proxy.

apt-cacher-ng has a second mode of operation where you can prefix
the source URL with the proxy URL, i.e.

deb http://proxyhost:3142/deb.debian.org/debian unstable main

Maybe we could introduce this as an "official" APT proxy mode, where
http(s)://REPO gets replaced by http://PROXY_URL/REPO (and the proxy
can decide whether or not to fetch via HTTPS as an implementation
detail)?


I'd much rather see something more like I proposed earlier (splitting the 
selection of what suites/components to install from the policy of how to 
obtain them) rather than further layering+confusing these two concepts 
within sources.list.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Timo Röhling]

* Michael Stone  [2021-09-08 19:25]:
I think the issue isn't certificate validation, it's that https proxy 
requests are made via CONNECT rather than GET. You could theoretically 
rewrite the proxy mechanism to MITM the CONNECT, but that wouldn't be 
a drop-in replacement. I suppose you could instead add an apt option 
to pass the https request to the proxy via GET instead of using 
CONNECT, but I think that also won't necessarily work on an existing 
proxy.

apt-cacher-ng has a second mode of operation where you can prefix
the source URL with the proxy URL, i.e.

deb http://proxyhost:3142/deb.debian.org/debian unstable main

Maybe we could introduce this as an "official" APT proxy mode, where
http(s)://REPO gets replaced by http://PROXY_URL/REPO (and the proxy
can decide whether or not to fetch via HTTPS as an implementation
detail)?

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Eduard Bloch]

Hallo,
* Michael Stone [Wed, Sep 08 2021, 07:25:26PM]:
> On Wed, Sep 08, 2021 at 03:56:14PM +0200, Ansgar wrote:
> > On Wed, 2021-09-08 at 15:41 +0200, Helmut Grohne wrote:
> > > On Wed, Sep 08, 2021 at 02:01:03PM +0200, Ansgar wrote:
> > > > So what do you suggest then? Tech-ctte as with merged-/usr? Or a
> > > > GR? Or
> > > > something else?
> > >
> > > I propose that the proponents pay the cost. In this case, it is a bit
> > > unclear what that means precisely (which likely is the reason they
> > > haven't done it already). At the very least though, apt install
> > > auto-apt-proxy should continue to work on a default installation in a
> > > sensible way.
> >
> > I can file a bug for auto-apt-proxy to include an apt.conf snippet
> > saying
> >
> >  Acquire::https::Verify-Peer false;
> >
> > That clearly makes it work again
>
> I think the issue isn't certificate validation, it's that https proxy
> requests are made via CONNECT rather than GET. You could theoretically
> rewrite the proxy mechanism to MITM the CONNECT, but that wouldn't be a
> drop-in replacement. I suppose you could instead add an apt option to pass
> the https request to the proxy via GET instead of using CONNECT, but I think

Precisely. Current handling of HTTPS on a caching proxy is either
impossible or PITA for the user, as long as a such mixed behavior is not
configurable.  apt-cacher-ng works around that by telling users to
disguise https URLs as HTTP with a special marker for protocol switch
(ugly, I know).

Also keep in mind that it off-loads the encryption work to the proxy,
but that might be even intentional.

> that also won't necessarily work on an existing proxy.

Speaking at least for ACNG, my assumption was that it would support that
but I was wrong. TODO created,
https://salsa.debian.org/blade/apt-cacher-ng/-/issues/11 .

> If we're imagining apt options, something like
> Acquire::https::Force-Proxy-HTTP true;
> would probably be more useful for this specific case (not that I think it's
> a great idea--too much potential for surprise).

I would make it a list of trusted proxy hosts and a special value ALL.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994032 created.

Best regards,
Eduard.

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Fri, 2021-09-10 at 09:33 +0200, Helmut Grohne wrote:
> If
> we installed auto-apt-proxy by default, much of the local caching
> would
> just work.

If you push for a local caching method to be used by default, apt
should always request (In)Release.gpg from a regular mirror (not auto-
discovered local cache), preferably via HTTPS; for subsequent data
(which apt can verify via (In)Release) a local mirror can be used,
falling back to the regular mirror when the data provided by the local
cache is not correct for any reason.

Especially at BSPs where people are likely to bootstrap new
environments (via debootstrap, for example for building packages) we
would allow downgrade attacks otherwise: (In)Release for stable
releases has no Valid-Until, so any initial (In)Release file can be
substituted by the cache operator for an older one which then refers to
known-vulnerable packages. (And I'm not sure debootstrap even checks
Valid-Until.)

Ansgar

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

On Wed, Sep 08, 2021 at 07:12:18PM -0400, Michael Stone wrote:
> Why not simply automate setting it at install time using preseed? I'm
> honestly not sure who the target audience for auto-apt-proxy is--apparently
> someone who has an infrastructure including a proxy, possibly the ability to
> set dns records, etc., but can't change defaults at install time or via some
> sort of runtime configuration management?

I believe that the target audience is non-tech end-users. Most
orgnizations already optimized their way of downloading .debs via some
way (e.g. auto-apt-proxy) or another. As you point out, organizations
are easily able to deviate from defaults. They're not our primary target
with defaults.

Laptops of end-user systems are the target, but also developers. When
people gather at a place (conference, hackspace, private meetup, etc.)
downloading of .debs should just work quickly by default. Many such
sites could easily provide a local cache and a number even do. BSPs tend
to have a blackboard with information including the local mirror to use.
Seriously, how many people change their mirror when they go to a BSP? If
we installed auto-apt-proxy by default, much of the local caching would
just work.

The thing we seem to be disagreeing on is what is more important? https
by default or quick and efficient downloads? Some may think that our
CDN can handle the load just fine and the effects of caching are
peanuts. I can attest that those peanuts are 4TB/year (equivalent to 8
days waiting for downloads) for me.

I see that we've given up on a global network of independently-managed
mirrors and that CDNs are way easier at this time. While initially CDNs
had bad reliability issues, those have completely vanished in my
experience. On the other hand, local caching still outperforms CDNs by a
factor of 10 or so. I'd love to make it the default.

Helmut

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Paul Wise]

On Thu, Sep 9, 2021 at 6:03 PM Simon Richter wrote:

> Another important argument is that it creates a dependency on
> third-party commercial CDNs, and their *continued* sponsorship.

This dependency on external providers is unavoidable, Debian
definitely cannot afford to run our own CDN at the scale needed to
support our existing userbase. For example the security mirrors
struggled with Linux kernel security updates, so security.d.o switched
to a commercial CDN. Also, we are dependent on continued sponsorship
for all of our infrastructure, paying for all of our hosting is likely
not feasible.

https://wiki.debian.org/ExternalEntities

> Debian is very conservative when spending money and generally shies away
> from recurring expenses because we do not want to find us in a situation
> where we are dependent on an external entity making a timely donation in
> order to keep operations running, so I wonder why we are that accepting
> of it in one of our core services, and I certainly don't think we should
> be adding additional roadblocks should we ever need to find an alternative.

DSA setup the CDN provider solution to give the Debian userbase a
better experience than having to choose a mirror and a better
experience than httpredir.d.o's redirect method. We have multiple CDN
providers to mitigate the dependency, and other providers who we
aren't yet using that are offering service. So, as much as I dislike
CDNs as a concept, I recognise that we currently need them and think
that we are able to handle loss of a CDN provider or two.

> We have a (crude) load-balancing framework in infrastructure we control
> that can point requests towards a set of untrusted mirrors, and while
> it's nice that we don't *need* to use this fallback plan, it is
> reassuring it is there.

httpredir.d.o no longer exists, it points at deb.d.o, so it would have
to be rebuilt if we were to need to switch away from CDNs.

Personally I'd like to see a larger variety of Debian delivery
mechanisms; copy Debian/snapshot to archive.org, create a multi-distro
FLOSS CDN, bring back httpredir, DebTorrent and apt-p2p, add an i2p
mirror, use IPFS and content oriented networking etc. Michael Stone's
apt://debian idea seems like a good way to move in that direction
while adding protocol agility.

> If they ask why we're not using HTTPS, yes: it helps clear up the
> misconception that anything with an "s" in it is secure and can be trusted.

The volume of questions about missing https means that it is more
efficient to just use https than to have to reply to new questions
about it.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 04.09.21 22:12, Hideki Yamane wrote:


The TLS layer is not part of the security model, so we'd be teaching
users to look for the wrong thing, kind of like the "encrypted with SSL"
badges on web pages in the 90ies.



  Is there any strong reason to use HTTP than HTTPS now?


The strongest reason (IMO) in favor of unencrypted transmission is that 
it doesn't introduce a policy decision. The package "ca-certificates" 
must be installed and a checkmark set for the "mozilla/ISRG_Root_X1.crt" 
certificate, otherwise updates will break.


If we want to have HTTPS as default, we need additional logic to make 
sure certificates are installed and cannot be deinstalled (so Essential 
or a strong dependency chain from an Essential package) and that the 
certificate cannot be deactivated, or apt needs its own repository of 
trusted certificates.


With the current Docker images:

$ docker run --rm -it debian:bullseye
root@32529bf86cd3:/# sed -i -e s/http/https/ /etc/apt/sources.list
root@32529bf86cd3:/# apt update
Err:1 https://deb.debian.org/debian bullseye InRelease
  Certificate verification failed: The certificate is NOT trusted. The 
certificate issuer is unknown.  Could not handshake: Error in the 
certificate verification. [IP: 199.232.138.132 443]
Err:2 https://security.debian.org/debian-security bullseye-security 
InRelease
  Certificate verification failed: The certificate is NOT trusted. The 
certificate issuer is unknown.  Could not handshake: Error in the 
certificate verification. [IP: 151.101.66.132 443]

Err:3 https://deb.debian.org/debian bullseye-updates InRelease
  Certificate verification failed: The certificate is NOT trusted. The 
certificate issuer is unknown.  Could not handshake: Error in the 
certificate verification. [IP: 199.232.138.132 443]


So changing the default is not sufficient here, we'd need a lot of 
additional work and testing to ensure this works for everyone, not just 
the desktop users.


Another important argument is that it creates a dependency on 
third-party commercial CDNs, and their *continued* sponsorship.


Debian is very conservative when spending money and generally shies away 
from recurring expenses because we do not want to find us in a situation 
where we are dependent on an external entity making a timely donation in 
order to keep operations running, so I wonder why we are that accepting 
of it in one of our core services, and I certainly don't think we should 
be adding additional roadblocks should we ever need to find an alternative.


We have a (crude) load-balancing framework in infrastructure we control 
that can point requests towards a set of untrusted mirrors, and while 
it's nice that we don't *need* to use this fallback plan, it is 
reassuring it is there.



  Should we teach all our users (including non-tech) about "Secure APT"
  mechanism?


If they ask why we're not using HTTPS, yes: it helps clear up the 
misconception that anything with an "s" in it is secure and can be trusted.


Anyone who configures an additional source needs to know how the 
authentication mechanism works anyway, so we're not gaining anything 
there either.


   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Timo Röhling]

* Michael Stone  [2021-09-09 09:05]:
Because the controversy concerning changing the default is over 
whether it's reasonable for someone using auto-apt-proxy to have to 
manage additional configuration settings.

Ah, I understand your point now and I agree.
It would be an inconvenience, yes, not but much more.

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Thu, Sep 09, 2021 at 02:54:21PM +0200, Timo Röhling wrote:

* Michael Stone  [2021-09-09 08:32]:
I'm honestly not sure who the target audience for auto-apt-proxy 
is--apparently someone who has an infrastructure including a 
proxy, possibly the ability to set dns records, etc., but can't 
change defaults at install time or via some sort of runtime 
configuration management?

The same reason you might want to deploy WPAD instead of preconfiguring
proxy settings in web browsers: flexibility. For example, we use
auto-apt-proxy for laptops which roam between different networks.
It's simple to configure, has virtually no maintenance overhead and
degrades gracefully.
None of that speaks to whether an organization that deploys such a 
thing has the ability to manage other configuration settings, even 
if for some settings there's a desire for additional flexibility.

I don't understand your point.

You asked for a target audience for auto-apt-proxy. I gave you a
legitimate (and real-world) example for such a deployment. Why
should it matter whether or not an organization has other
configuration facilities?


Because the controversy concerning changing the default is over whether 
it's reasonable for someone using auto-apt-proxy to have to manage 
additional configuration settings. If the target audience is someone who 
has the ability to manage the infrastructure required by auto-apt-proxy 
but not the ability to manage anything else then I guess it's a valid 
criticism (but I have trouble envisioning that). If the auto-apt-proxy 
target audience can manage both the proxy infrastructure *and* 
sources.list (either at install time or run time) then the criticism is 
basically academic and not generally a real-world issue.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Timo Röhling]

* Michael Stone  [2021-09-09 08:32]:
I'm honestly not sure who the target audience for auto-apt-proxy 
is--apparently someone who has an infrastructure including a 
proxy, possibly the ability to set dns records, etc., but can't 
change defaults at install time or via some sort of runtime 
configuration management?

The same reason you might want to deploy WPAD instead of preconfiguring
proxy settings in web browsers: flexibility. For example, we use
auto-apt-proxy for laptops which roam between different networks.
It's simple to configure, has virtually no maintenance overhead and
degrades gracefully.
None of that speaks to whether an organization that deploys such a 
thing has the ability to manage other configuration settings, even if 
for some settings there's a desire for additional flexibility.

I don't understand your point.

You asked for a target audience for auto-apt-proxy. I gave you a
legitimate (and real-world) example for such a deployment. Why
should it matter whether or not an organization has other
configuration facilities?

As another example, nobody says: "the target audience for DHCP is an
organization who has an infrastructure with full control over a
network, including IP address management, but apparently can't
configure IP addresses at install time or via some sort of runtime
configuration management" and concludes that DHCP is useless.

auto-apt-proxy (or DHCP for that matter) *is* the runtime
configuration management, and a quite efficient one at that.

Cheers
Timo


--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Thu, Sep 09, 2021 at 11:54:44AM +0530, Pirate Praveen wrote:

Why can't auto-apt-proxy ask this as a debconf question? I also like 
auto-apt-proxy but I agree with  this, someone needing auto-apt-proxy should be 
able to change the default as well.


I don't really see why adding another debconf question would be better 
than just preseeding the existing one.


The only thing I could see that would be a net gain would be to 
generalizes sources.list more. Instead of having a user select a 
specific protocol and path, allow the user to just select high-level 
objects. Make this a new pseudo-protocol for backward compatibility, and 
introduce something like

  deb apt:// suite component[s]
so the default could be something like
  deb apt:// bullseye main
  deb apt:// bullseye/updates main
then the actual protocols, servers, and paths could be managed by 
various plugins and overridden by configuration directives in 
apt.conf.d. For existing configurations it's a no-op but it allows more 
flexibility & new plugins/protocols in the future without having to 
micromanage sources.list. If someone wants to use tor by default but 
fall back to https if it's unreachable, they can do that. If someone 
wants to use a local proxy via http but https if they're not on their 
local network, they can do that. New installations could default to 
https, existing installations can keep doing their thing, and a plugin 
like auto-apt-proxy can override defaults to do something more 
complicated, using more policy-friendly .d configurations rather than 
figuring out a way to rewrite some other package's configuration file.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Thu, Sep 09, 2021 at 08:36:28AM +0200, Timo Röhling wrote:

* Michael Stone  [2021-09-08 19:12]:
Why not simply automate setting it at install time using preseed? 
I'm honestly not sure who the target audience for auto-apt-proxy 
is--apparently someone who has an infrastructure including a proxy, 
possibly the ability to set dns records, etc., but can't change 
defaults at install time or via some sort of runtime configuration 
management?

The same reason you might want to deploy WPAD instead of preconfiguring
proxy settings in web browsers: flexibility. For example, we use
auto-apt-proxy for laptops which roam between different networks.
It's simple to configure, has virtually no maintenance overhead and
degrades gracefully.


None of that speaks to whether an organization that deploys such a thing 
has the ability to manage other configuration settings, even if for 
some settings there's a desire for additional flexibility.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Timo Röhling]

* Michael Stone  [2021-09-08 19:12]:
Why not simply automate setting it at install time using preseed? I'm 
honestly not sure who the target audience for auto-apt-proxy 
is--apparently someone who has an infrastructure including a proxy, 
possibly the ability to set dns records, etc., but can't change 
defaults at install time or via some sort of runtime configuration 
management?

The same reason you might want to deploy WPAD instead of preconfiguring
proxy settings in web browsers: flexibility. For example, we use
auto-apt-proxy for laptops which roam between different networks.
It's simple to configure, has virtually no maintenance overhead and
degrades gracefully. 


Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Pirate Praveen]

2021, സെപ്റ്റംബർ 9 4:42:18 AM IST, Michael Stone ൽ എഴുതി
>On Wed, Sep 08, 2021 at 01:09:13PM +0200, Helmut Grohne wrote:
>>Enabling https by default quite simply breaks the simple recipe of
>>installing auto-apt-proxy. Would you agree with auto-apt-proxy's
>>postinst automatically editing your sources.list to drop the s out of
>>https? The answer repeatedly given in this thread to do so manually is
>>very unsatisfying.
>
>Why not simply automate setting it at install time using preseed? I'm 
>honestly not sure who the target audience for auto-apt-proxy 
>is--apparently someone who has an infrastructure including a proxy, 
>possibly the ability to set dns records, etc., but can't change defaults 
>at install time or via some sort of runtime configuration management?
>

Why can't auto-apt-proxy ask this as a debconf question? I also like 
auto-apt-proxy but I agree with  this, someone needing auto-apt-proxy should be 
able to change the default as well.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Timothy M Butterworth]

All,

I am against automatically setting HTTPS. Their should be an option in
the installer to set or unset HTTPS while configuring the mirror! I
like a lot of folks am on a metered internet connection with a UTM
proxy firewall. I have multiple computers that need patched and only
having to download the package once is a great convenience to both
data usage and download time as my internet is not fast 4G LTE usually
only operating at 3G speed.

Tim

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Wed, Sep 08, 2021 at 03:56:14PM +0200, Ansgar wrote:

On Wed, 2021-09-08 at 15:41 +0200, Helmut Grohne wrote:

On Wed, Sep 08, 2021 at 02:01:03PM +0200, Ansgar wrote:
> So what do you suggest then? Tech-ctte as with merged-/usr? Or a
> GR? Or
> something else?

I propose that the proponents pay the cost. In this case, it is a bit
unclear what that means precisely (which likely is the reason they
haven't done it already). At the very least though, apt install
auto-apt-proxy should continue to work on a default installation in a
sensible way.


I can file a bug for auto-apt-proxy to include an apt.conf snippet
saying

 Acquire::https::Verify-Peer false;

That clearly makes it work again


I think the issue isn't certificate validation, it's that https proxy 
requests are made via CONNECT rather than GET. You could theoretically 
rewrite the proxy mechanism to MITM the CONNECT, but that wouldn't be a 
drop-in replacement. I suppose you could instead add an apt option to 
pass the https request to the proxy via GET instead of using CONNECT, 
but I think that also won't necessarily work on an existing proxy.


If we're imagining apt options, something like 
Acquire::https::Force-Proxy-HTTP true;
would probably be more useful for this specific case (not that I think 
it's a great idea--too much potential for surprise).

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Michael Stone]

On Wed, Sep 08, 2021 at 01:09:13PM +0200, Helmut Grohne wrote:

Enabling https by default quite simply breaks the simple recipe of
installing auto-apt-proxy. Would you agree with auto-apt-proxy's
postinst automatically editing your sources.list to drop the s out of
https? The answer repeatedly given in this thread to do so manually is
very unsatisfying.


Why not simply automate setting it at install time using preseed? I'm 
honestly not sure who the target audience for auto-apt-proxy 
is--apparently someone who has an infrastructure including a proxy, 
possibly the ability to set dns records, etc., but can't change defaults 
at install time or via some sort of runtime configuration management?

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Wed, 2021-09-08 at 15:41 +0200, Helmut Grohne wrote:
> On Wed, Sep 08, 2021 at 02:01:03PM +0200, Ansgar wrote:
> > So what do you suggest then? Tech-ctte as with merged-/usr? Or a
> > GR? Or
> > something else?
> 
> I propose that the proponents pay the cost. In this case, it is a bit
> unclear what that means precisely (which likely is the reason they
> haven't done it already). At the very least though, apt install
> auto-apt-proxy should continue to work on a default installation in a
> sensible way.

I can file a bug for auto-apt-proxy to include an apt.conf snippet
saying

  Acquire::https::Verify-Peer false;

That clearly makes it work again: you ask for auto-apt-proxy users to
have connections that can be impersonated by a man in the middle by
default. The above setting does that.

Not verifying certificates for some users seems better than having all
users not verify certificates (as no https is used at all).


> In
> the absence of reason not to use https, https should be preferred. As
> it
> happens, we figured a reason not to use https.

I can find a reason not to use https for any protocol (some sites want
to inspect/cache all traffic) :-)


Ansgar

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

On Wed, Sep 08, 2021 at 02:01:03PM +0200, Ansgar wrote:
> So what do you suggest then? Tech-ctte as with merged-/usr? Or a GR? Or
> something else?

I propose that the proponents pay the cost. In this case, it is a bit
unclear what that means precisely (which likely is the reason they
haven't done it already). At the very least though, apt install
auto-apt-proxy should continue to work on a default installation in a
sensible way.

> >  * Concerns are ignored. <- This is where we are with https-default.
> 
> It's also where we are with keep-http-as-default.

I don't think https resolves any concerns. It's merely best-practice. In
the absence of reason not to use https, https should be preferred. As it
happens, we figured a reason not to use https.

> > Change has a cost. I do not want to pay the cost for either of these
> > changes.
> 
> Then we could never change anything.

Untrue. You get to choose which changes you want to pay the cost for.
For instance, I want Debian to be cross buildable and bootstrappable.
Holger, Mattia and a few others want Debian to be reproducible. You
don't get to pay the cost for those changes. Change is possible in a way
that limits cost for uninterested people. The contentious changes are
the ones where the initiators fail to pay the cost.

> To keep up with merged-/usr: keeping non-merged-/usr also has a cost.
> Nobody wants to pay the cost for it.

That is very true. With merged-/usr, I suppose most grief arises from
the way the transition was (not) planned and only a minority takes issue
with the goal.

Helmut

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Wed, 2021-09-08 at 13:13 +0100, Tim Woodall wrote:
> This is a bit tongue in cheek, but how about these sites where the
> .debs are downloaded from publish their *private* key? They openly
> accept that anyone can MITM them.

If you have access to the private key, you can request the CA to revoke
the certificate:

+---
| 4.9.1.1 Reasons for Revoking a Subscriber Certificate
|
| The CA SHALL revoke a Certificate within 24 hours if one or more of
| the following occurs:
| [...]
| 3. The CA obtains evidence that the Subscriber’s Private Key
| corresponding to the Public Key in the Certificate suffered a Key
| Compromise
+---[ https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.8.0.pdf ]

So that would not be helpful.

Ansgar

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Tim Woodall]

On Wed, 8 Sep 2021, Helmut Grohne wrote:


I do see the advantages of using https. I do not see how to not make it
happen without breaking relevant use cases. Same with the /usr-merge. I
do see the advantages. I've stopped counting the things that broke. Most
recent one is the uucp FTBFS. Change has a cost. I do not want to pay
the cost for either of these changes.



This is a bit tongue in cheek, but how about these sites where the .debs
are downloaded from publish their *private* key? They openly accept that
anyone can MITM them.

That way people who want to see "https" can see it. And people who want
the benefits of http can, with a bit of work, simulate that.

It also nicely addresses my concern which is that the next demand will
be to drop http (because when you visit the site with a webbrowser users
start getting a warning that the site is also available over http or
something like that because the google/firefox dream seems to be that
you cannot use http even where https doesn't add anything.)

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Wed, 2021-09-08 at 13:53 +0200, Helmut Grohne wrote:
> On Wed, Sep 08, 2021 at 01:37:37PM +0200, Ansgar wrote:
> > Maybe we should just find out who is responsible for this decision
> > and
> > reassign the bug to them.  The installer team maintaining d-i and
> > debootstrap or the mirror team seem reasonable choices?
> 
> We've already tried that approach on the /usr-merge and the resulting
> transition is miserable. Let's not repeat that mistake.

So what do you suggest then? Tech-ctte as with merged-/usr? Or a GR? Or
something else?

> It's the same pattern actually:
>  * People propose a change that does have positive effects, though
> some
>    find the positive effects unimportant.
>  * Other people disagree and raise concerns.
>  * Concerns are ignored. <- This is where we are with https-default.

It's also where we are with keep-http-as-default.

> Change has a cost. I do not want to pay the cost for either of these
> changes.

Then we could never change anything.

To keep up with merged-/usr: keeping non-merged-/usr also has a cost.
Nobody wants to pay the cost for it.

Ansgar

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

On Wed, Sep 08, 2021 at 01:37:37PM +0200, Ansgar wrote:
> Maybe we should just find out who is responsible for this decision and
> reassign the bug to them.  The installer team maintaining d-i and
> debootstrap or the mirror team seem reasonable choices?

We've already tried that approach on the /usr-merge and the resulting
transition is miserable. Let's not repeat that mistake.

It's the same pattern actually:
 * People propose a change that does have positive effects, though some
   find the positive effects unimportant.
 * Other people disagree and raise concerns.
 * Concerns are ignored. <- This is where we are with https-default.
 * Change is being implemented anyway.
 * Stuff breaks. <- This is where we are with /usr-merge.

This is frustrating.

I do see the advantages of using https. I do not see how to not make it
happen without breaking relevant use cases. Same with the /usr-merge. I
do see the advantages. I've stopped counting the things that broke. Most
recent one is the uucp FTBFS. Change has a cost. I do not want to pay
the cost for either of these changes.

Helmut

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Wed, 2021-09-08 at 13:09 +0200, Helmut Grohne wrote:
> On Thu, Sep 02, 2021 at 10:22:15AM +0900, Hideki Yamane wrote:
> >  Some users want proxy but they can configure their settings.
> >  So just change "default setting for {deb,security}.debian.org"
> >  is not so harmful, IMO. 
> 
> I fear you are putting this upside down. In reality, some sites (not
> users) want their users to use their local cache (transparently or
> not).

Then have the users install the site's CA authority that allows
inspecting and caching HTTPS traffic.


> Unfortunately, I don't see consensus for this, but at the same time I
> neither see consensus for enabling https by default. It's a matter
> that
> keeps popping up and people disagreeing on over and over again. The
> one
> thing that we have clearly understood at this point is that one size
> does not fit everyone. Either way makes some people unhappy.

Maybe we should just find out who is responsible for this decision and
reassign the bug to them.  The installer team maintaining d-i and
debootstrap or the mirror team seem reasonable choices?

Ansgar

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

Hi,

On Thu, Sep 02, 2021 at 10:22:15AM +0900, Hideki Yamane wrote:
>  Some users want proxy but they can configure their settings.
>  So just change "default setting for {deb,security}.debian.org"
>  is not so harmful, IMO. 

I fear you are putting this upside down. In reality, some sites (not
users) want their users to use their local cache (transparently or not).

>  - Users can choose other mirror than https://deb.debian.org

As far as I can tell, most users don't want to make a choice here. They
want downloading packages to just work. Preferably fast. It is the
"fast" thing that you are breaking here.

>  - Caching .debs from security.debian.org is not so huge, I guess
>(maybe except linux-image).

Not sure why security.d.o is singled out here. The switch is only
reasonable on the whole or not at all. And there the whole volume of
packages counts.

Enabling https by default quite simply breaks the simple recipe of
installing auto-apt-proxy. Would you agree with auto-apt-proxy's
postinst automatically editing your sources.list to drop the s out of
https? The answer repeatedly given in this thread to do so manually is
very unsatisfying.

So I actually argue for installing auto-apt-proxy by default and inside
d-i. That is in direct conflict with the proposed change here.

Unfortunately, I don't see consensus for this, but at the same time I
neither see consensus for enabling https by default. It's a matter that
keeps popping up and people disagreeing on over and over again. The one
thing that we have clearly understood at this point is that one size
does not fit everyone. Either way makes some people unhappy.

Helmut

Bug#992692: general: Use https for {deb,security}.debian.org by default

[David Kalnischkies]

On Fri, Sep 03, 2021 at 02:42:29AM +, Paul Wise wrote:
> httpredir.d.o was an alternative CDN-like thing that was based on HTTP
> redirects to the mirror network. It had lots of problems, but now that
> we have a mirror checker and zzz-dists, perhaps it could work better.
> The mirror:// method in apt has probably improved since then too.

If you wanted to bring back a httpredir-like¹ you are better of to
combine both approaches as in: Have apt request a list of mirrors to use
via mirror(+https) and have the server generate that list based on the
requester as that gives you the "regional" mirrors as did httpredir while
solving the major grip it had by having a list of mirrors to use, rather
than one potentially non-working slightly outdated partial mirror (and
the httpredir service is contacted by each client once rather than for
each individual file to then be redirected elsewhere).

Obviously, that approach is only workable if you are actually using
libapt tools. Most debootstrap implementations couldn't really use that
which might or might not be a problem for a given use case. Such
a service would also have a hard time to 'redirect' you to a local
mirror in your network (compared to an 'official' region one).


So that isn't really what seems to be the main worry here:
https prevents MitM attacks including the friendly MitM ones like the
local network at home/at DebConf telling my laptop that there is an
on-site mirror or not telling at all and just proxy transparently the
entire network.

The later seems done for in a https-world, but the former might be
somewhat salvageable: We will have to get the Release² file(s) from the
repo defined in the sources, but the index files and debs after that are
a fairer game to get from elsewhere as they are either identical with
what the defined source would have provided or a hard error.
That still violates the privacy guarantees https has (assuming it does),
so that would still need to be opt-in/out, but that is a one time choice
per machine and could be similar in style to auto-apt-proxy.

Anyway, implementation wise apt could tell $MAGIC which repo it is
interested in (by Origin & Label) and would in return get a list of
mirrors as apt-transport-mirror would. apt would then add the original
source as least priority fallback and proceed with that list for this
source.
I say $MAGIC as I don't want apt to hard code the specifics of how to
get the list, similar to how it is agnostic to how a proxy is currently
picked up, as I could envision different implementations depending on
use cases.

That is different to just using apt-transport-mirror directly in the
sources in so far as the provider of the list remains untrusted (beside
that nobody is constantly editing their sources to adapt to the local
network the machine currently resides in).


Relatively quickly thought up, probably full of holes and not implemented
at all in apt so far, but if someone thinks that might work feel free to
report as a feature request against apt and I will see what I can do
from the apt side. It at least seems slightly more workable than hoping
to prevent https – which might have just as dubious a chance to succeed
as https has to factually improve security in terms of apt. 


> Maybe http redirects to local mirrors could be feasible again, but it
> would take a lot of work.

fwiw: apt does not allow https to http redirects (some https repos
ran into this in the past like those hosted on sourceforge until they
fixed their https 'everywhere' configuration). In this regard apt is
stricter than a normal webbrowser (a mirror list acquired via https can
redirect to http mirrors though, but see the man pages for details).


Best regards

David Kalnischkies


¹ which deb.d.o sort of is just that it is nowadays done via SRV instead
  of an explicit HTTP redirect – and that only one mirror is in the list
  rather than multiple httpredir had picked one to redirect to.

² The main security benefit of https for apt is that you can't fiddle
  with the Release file, mostly in terms of sending an older one (in
  the limits of Valid-Until if used). It is also minor in size compared
  to the indexes and especially the debs, so caching them is not much of
  a concern (if a cacher was even doing it, it probably shouldn't).


signature.asc
Description: PGP signature

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Hideki Yamane]

On Thu, 2 Sep 2021 21:26:11 +0200
Simon Richter  wrote:
> The TLS layer is not part of the security model, so we'd be teaching 
> users to look for the wrong thing, kind of like the "encrypted with SSL" 
> badges on web pages in the 90ies.

 Is there any strong reason to use HTTP than HTTPS now?
 Should we teach all our users (including non-tech) about "Secure APT"
 mechanism?


 And I said about only deb.debian.org and security.debian.org, and
 just "default" - it means it does provide http access too.



-- 
Regards,

 Hideki Yamane henrich @ debian.org/iijmio-mail.jp

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Philipp Kern]

Hi,

On 03.09.21 13:11, Simon Richter wrote:
[Revocation mechanism]

If we don't have one, shouldn't we worry more about that given the
widespread use of TLS?
We have a big hammer, shipping a new ca-certificates package. If we want 
something that only affects apt, but not other packages, that mechanism 
doesn't exist yet.


I think that's an interesting point, not just for revocation. There are 
forces pushing for more agility, switching out roots of trust more 
frequently. So for very old releases, you usually had the signing key of 
the next release on disk, so you could move to the next release. In this 
case you sort of risk not having the TLS authority on disk to make that 
happen. And of course we need to track what the authorities are doing 
that our frontends are using (e.g. [1] around how to deal with old 
Android devices).


But then I'm not sure how much we need to care about ancient releases 
that are out of security support. We would need to commit to regularly 
update the certificate bundle, though.


To your other point: I don't think managing trust into individual CAs 
will scale. We cannot really anticipate which CAs we are going to use in 
the future.


Kind regards
Philipp Kern

[1] https://letsencrypt.org/2020/12/21/extending-android-compatibility.html

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Fri, 2021-09-03 at 13:11 +0200, Simon Richter wrote:
> > >    - If I deselect all CAs in the configuration dialog of the
> > > ca-certificates package, what mechanism will allow apt to work?
> 
> > If people intentionally detrust them, they have to deal with the
> > fallout.
> 
> So this introduces a policy that users need to mark X.509 CAs as
> trusted?

No.

> > People can also detrust Debian's OpenPGP signing keys; it's
> > not much different.
> 
> The Debian signing keys have separate trust setup, and are trusted
> for nothing but APT updates.
> 
> If we wanted the same for X.509, we'd need /etc/apt/ca-certificates
> or 
> something such, and apt to configure the list of accepted CAs
> explicitly 
> in the TLS layer rather than using the default settings.

People who only want to trust specific CAs for APT can do that. APT has
a CaPath setting.

> > Accessing www.debian.org will also not work on such systems (and
> > unlike
> > deb.d.o that does not even offer non-https). It's not Debian's
> > problem.
> 
> There are a lot of systems out there that have no need to access 
> www.debian.org, but do need to access deb.debian.org.

And nothing stops them from doing so.

> > >    - Do we want to pin the certificate provider for Debian
> > > mirrors, in
> > > the knowledge that we want to be bound to this provider for
> > > several
> > > years, do we want any "root" CA to be able to provide a trust
> > > anchor?
> 
> > Probably not?
> 
> So what do we want then? A list of root CAs that users have to mark
> as  trusted, possibly with an "are you sure?" dialog in ca-
> certificates?

We already have that.

> This isn't a simple change of default, because this simple change
> pulls in a lot of dependencies.

ca-certificates should already be installed by default (it is Priority:
standard and recommended by apt).

> That users can override the default means adding another work step
> for users, either a manual step that needs to be performed after a
> manual installation, or an automated step that needs to be integrated
> into users' deployment processes.

I don't see the problem? Currently we add another work step for users
that want to use https.

> > If we don't have one, shouldn't we worry more about that given the
> > widespread use of TLS?
> 
> We have a big hammer, shipping a new ca-certificates package. If we
> want something that only affects apt, but not other packages, that
> mechanism doesn't exist yet.

If a CA is untrustworthy, I don't think we would only want to detrust
it for apt's https method. So I see no problem.

> 
> It's not about what I like, but on what external services we want to
> depend.

So your concern is about Debian providing the deb.debian.org service at
all? That seems unrelated to the https or not question.

Ansgar

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 02.09.21 23:02, Ansgar wrote:


As it is now, I can install a Debian system where no X.509
certificate authorities are trusted.



That doesn't change with the proposal?



   - If I deselect all CAs in the configuration dialog of the
ca-certificates package, what mechanism will allow apt to work?



If people intentionally detrust them, they have to deal with the
fallout.


So this introduces a policy that users need to mark X.509 CAs as trusted?


People can also detrust Debian's OpenPGP signing keys; it's
not much different.


The Debian signing keys have separate trust setup, and are trusted for 
nothing but APT updates.


If we wanted the same for X.509, we'd need /etc/apt/ca-certificates or 
something such, and apt to configure the list of accepted CAs explicitly 
in the TLS layer rather than using the default settings.



Accessing www.debian.org will also not work on such systems (and unlike
deb.d.o that does not even offer non-https). It's not Debian's problem.


There are a lot of systems out there that have no need to access 
www.debian.org, but do need to access deb.debian.org.



   - Do we want to pin the certificate provider for Debian mirrors, in
the knowledge that we want to be bound to this provider for several
years, do we want any "root" CA to be able to provide a trust anchor?



Probably not?


So what do we want then? A list of root CAs that users have to mark as 
trusted, possibly with an "are you sure?" dialog in ca-certificates?


This isn't a simple change of default, because this simple change pulls 
in a lot of dependencies. That users can override the default means 
adding another work step for users, either a manual step that needs to 
be performed after a manual installation, or an automated step that 
needs to be integrated into users' deployment processes.



   - Is there a revocation mechanism by which we can mark "root" CAs
as
untrustworthy?



If we don't have one, shouldn't we worry more about that given the
widespread use of TLS?


We have a big hammer, shipping a new ca-certificates package. If we want 
something that only affects apt, but not other packages, that mechanism 
doesn't exist yet.



Do we have a revocation mechanism by which we can mark OpenPGP signing
keys as untrustworthy (say for apt)?


Yes, by shipping an update.


   - do we have a contingency plan if deb.debian.org hosting on Fastly
is no longer feasible?



As far as I know there is also at least https://cdn-aws.deb.debian.org/
if you don't like Fastly.


It's not about what I like, but on what external services we want to depend.

   Simon



OpenPGP_signature
Description: OpenPGP digital signature

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Paul Wise]

On Thu, Sep 2, 2021 at 9:06 PM Ansgar wrote:

> Accessing www.debian.org will also not work on such systems (and unlike
> deb.d.o that does not even offer non-https). It's not Debian's problem.

The Tor onion services offer alternatives to the https PKI:

https://onion.debian.org/

> Is replacing deb.d.o by a non-CDN feasible?

security.d.o mirrors were getting overwhelmed after Linux kernel
updates, which is why that switched to the Fastly CDN, so probably
not. Also, the entire point of the deb.d.o domain is that it be backed
by a CDN.

httpredir.d.o was an alternative CDN-like thing that was based on HTTP
redirects to the mirror network. It had lots of problems, but now that
we have a mirror checker and zzz-dists, perhaps it could work better.
The mirror:// method in apt has probably improved since then too.
Maybe http redirects to local mirrors could be feasible again, but it
would take a lot of work.

https://mirror-master.debian.org/status/mirror-hierarchy.html

> As far as I know there is also at least https://cdn-aws.deb.debian.org/
> if you don't like Fastly.

And there are other CDNs that could potentially be used.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Thu, 2021-09-02 at 21:26 +0200, Simon Richter wrote:
> As it is now, I can install a Debian system where no X.509
> certificate authorities are trusted.

That doesn't change with the proposal?

>   - If I deselect all CAs in the configuration dialog of the 
> ca-certificates package, what mechanism will allow apt to work?

If people intentionally detrust them, they have to deal with the
fallout. People can also detrust Debian's OpenPGP signing keys; it's
not much different.

Accessing www.debian.org will also not work on such systems (and unlike
deb.d.o that does not even offer non-https). It's not Debian's problem.

>   - Do we want to pin the certificate provider for Debian mirrors, in
> the knowledge that we want to be bound to this provider for several 
> years, do we want any "root" CA to be able to provide a trust anchor?

Probably not?

>   - Is there a revocation mechanism by which we can mark "root" CAs
> as 
> untrustworthy?

If we don't have one, shouldn't we worry more about that given the
widespread use of TLS?

Do we have a revocation mechanism by which we can mark OpenPGP signing
keys as untrustworthy (say for apt)?

>   - What does the UI look like if OSCP verification fails?
>   - How do mirror operators get a signed certificate?

Not all mirror operators have a TLS certificate. I don't think that was
proposed to be changed (see subject of the mail).

> I think we're adding a lot of complexity and external dependencies to
> the system here, which adds a lot of burden to mirror operators that 
> aren't large CDNs.

See above.

>   - do we have a contingency plan if deb.debian.org hosting on Fastly
> is no longer feasible?

Is replacing deb.d.o by a non-CDN feasible? If no, what does use of
https change?

As far as I know there is also at least https://cdn-aws.deb.debian.org/
if you don't like Fastly.

Ansgar

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Simon Richter]

Hi,

On 02.09.21 03:22, Hideki Yamane wrote:


  Providing "default secure setting" is good message to users.


The TLS layer is not part of the security model, so we'd be teaching 
users to look for the wrong thing, kind of like the "encrypted with SSL" 
badges on web pages in the 90ies.


We have our own PKI that is decoupled from the X.509 certificate 
infrastructure, and neither ascribes any trust in them nor depends on 
the availability of an external service.


As it is now, I can install a Debian system where no X.509 certificate 
authorities are trusted.


 - If I deselect all CAs in the configuration dialog of the 
ca-certificates package, what mechanism will allow apt to work?
 - Do we want to pin the certificate provider for Debian mirrors, in 
the knowledge that we want to be bound to this provider for several 
years, do we want any "root" CA to be able to provide a trust anchor?
 - Is there a revocation mechanism by which we can mark "root" CAs as 
untrustworthy?

 - What does the UI look like if OSCP verification fails?
 - How do mirror operators get a signed certificate?

I think we're adding a lot of complexity and external dependencies to 
the system here, which adds a lot of burden to mirror operators that 
aren't large CDNs. That may be acceptable for an entity like Ubuntu, who 
aren't dependent on donations, but we would be tied to the goodwill of 
CDN operators here, so:


 - do we wish to communicate that the existing mirrors outside 
deb.debian.org are somehow less "secure"?
 - do we have a contingency plan if deb.debian.org hosting on Fastly is 
no longer feasible?


   Simon

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-09-02 12:27:34 -0400 (-0400), Roberto C. Sánchez wrote:
[...]
> In this context, it might make sense to describe using HTTPS as
> the transport for APT operations is providing "default
> confidentiality".

Which, as similarly discussed, it really doesn't do either (because
of deterministic blob sizes for publicly served files, current SNI
implementations leaking hostnames, general state of the CA and CDN
industries...), so suggesting that may also give users a false
impression. If they really do need confidentiality of their package
installation, they're probably better off doing updates over Tor or
a some VPN which does batching/interleaving of bulk transfers with
some cover traffic, or keeping a private local package mirror.

But to a great extent the degree of confidentiality they can expect
really depends on who they're trying to hide their traffic from. If
their concern is that a company headquartered in the USA might be
tracking the packages they're downloading from deb.debian.org, then
that's already a possibility even with HTTPS: the site is currently
fronted by the Fastly CDN service which terminates TLS encryption
for those HTTPS requests in order to be able to cache them globally.
Of course, without a CDN, mirror site operators can track what
packages you're downloading from them over HTTPS as well.

More generally, what I'm saying is don't try to paint this change as
actually implementing any significant amount of new security or
privacy for Debian users, that would be disingenuous. Just say the
default is switching to HTTPS because that's what users, by and
large, expect today.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Roberto C . Sánchez]

On Thu, Sep 02, 2021 at 04:08:37PM +, Jeremy Stanley wrote:
> On 2021-09-02 10:22:15 +0900 (+0900), Hideki Yamane wrote:
> [...]
> >  Providing "default secure setting" is good message to users.
> [...]
> 
> As previously covered, I'd suggest steering clear of referring to
> this as adding "default security." That implies APT wasn't already
> effectively secure over plain HTTP, and may give a false impression
> that HTTPS is addressing gaps in the existing apt-secure design.
> 
> This change is more about recognizing HTTPS as the primary transport
> protocol for the modern Web, not sending mixed signals regarding the
> general security risks posed by plain HTTP when used for unrelated
> purposes, and no longer needing to repeatedly explain to users that
> Debian has gone to great lengths to implement package distribution
> security which doesn't really depend at all on transport layer
> encryption.

In this context, it might make sense to describe using HTTPS as the
transport for APT operations is providing "default confidentiality".

Regards,

-Roberto

-- 
Roberto C. Sánchez

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Jeremy Stanley]

On 2021-09-02 10:22:15 +0900 (+0900), Hideki Yamane wrote:
[...]
>  Providing "default secure setting" is good message to users.
[...]

As previously covered, I'd suggest steering clear of referring to
this as adding "default security." That implies APT wasn't already
effectively secure over plain HTTP, and may give a false impression
that HTTPS is addressing gaps in the existing apt-secure design.

This change is more about recognizing HTTPS as the primary transport
protocol for the modern Web, not sending mixed signals regarding the
general security risks posed by plain HTTP when used for unrelated
purposes, and no longer needing to repeatedly explain to users that
Debian has gone to great lengths to implement package distribution
security which doesn't really depend at all on transport layer
encryption.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Hideki Yamane]

Hi,

On Wed, 01 Sep 2021 07:46:07 -0700
Russ Allbery  wrote:
> >> I believe that the discussion has later identified that doing so would
> >> break squid-deb-proxy-client and auto-apt-proxy. Given that the
> >> security benefits are not strong (beyond embracing good habits), I
> >> think the reasonable thing to do is keep preferring http.
> 
> > That is an opt-in choice which likely only a small number of users use.
> > People wanting to use a caching proxy can just switch to http as part of
> > this choice; it doesn't seem a good reason to not use https by default
> > for all other users.
> 
> Completely agreed.

 Providing "default secure setting" is good message to users.


 Some users want proxy but they can configure their settings.
 So just change "default setting for {deb,security}.debian.org"
 is not so harmful, IMO. 

 - Users can choose other mirror than https://deb.debian.org
 - Caching .debs from security.debian.org is not so huge, I guess
   (maybe except linux-image).


-- 
Hideki Yamane 

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Russ Allbery]

Ansgar  writes:
> On Wed, 2021-09-01 at 11:15 +0200, Helmut Grohne wrote:

>> I believe that the discussion has later identified that doing so would
>> break squid-deb-proxy-client and auto-apt-proxy. Given that the
>> security benefits are not strong (beyond embracing good habits), I
>> think the reasonable thing to do is keep preferring http.

> That is an opt-in choice which likely only a small number of users use.
> People wanting to use a caching proxy can just switch to http as part of
> this choice; it doesn't seem a good reason to not use https by default
> for all other users.

Completely agreed.

>> Caching packages and transport level encryption are fundamentally
>> incompatible.

> No. You can explicitly configure apt to use a local caching mirror or
> use a trusted TLS certificate for the mirror the proxy impersonates.

Yes.  For example, the approach used by apt-cacher-ng works fine.
Explicitly opting in to a local cache seems desirable.

-- 
Russ Allbery (r...@debian.org)  

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Ansgar]

On Wed, 2021-09-01 at 11:15 +0200, Helmut Grohne wrote:
> I believe that the discussion has later identified that doing so
> would
> break squid-deb-proxy-client and auto-apt-proxy. Given that the
> security
> benefits are not strong (beyond embracing good habits), I think the
> reasonable thing to do is keep preferring http.

That is an opt-in choice which likely only a small number of users use.
People wanting to use a caching proxy can just switch to http as part
of this choice; it doesn't seem a good reason to not use https by
default for all other users.

> Caching packages and transport level encryption are fundamentally
> incompatible.

No. You can explicitly configure apt to use a local caching mirror or
use a trusted TLS certificate for the mirror the proxy impersonates.


Ansgar

Processed: Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Debian Bug Tracking System]

Processing control commands:

> tags -1 + moreinfo
Bug #992692 [general] general: Use https for {deb,security}.debian.org by 
default
Added tag(s) moreinfo.

-- 
992692: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992692
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Re: Bug#992692: general: Use https for {deb,security}.debian.org by default

[Helmut Grohne]

Control: tags -1 + moreinfo

On Sun, Aug 22, 2021 at 09:56:57PM +0900, Hideki Yamane wrote:
>  As we discussed on -devel(*), it seems that we can enable https for
>  {deb,security}.debian.org by default. With this bug report, I'll
>  collect related things and fix it.

I believe that the discussion has later identified that doing so would
break squid-deb-proxy-client and auto-apt-proxy. Given that the security
benefits are not strong (beyond embracing good habits), I think the
reasonable thing to do is keep preferring http.

Caching packages and transport level encryption are fundamentally
incompatible. Possibly it would make more sense to offer users a choice
between performance and privacy on installation?

Helmut

Bug#992692: general: Use https for {deb,security}.debian.org by default

[Hideki Yamane]

Package: general
Severity: wishlist

Dear developers,

 As we discussed on -devel(*), it seems that we can enable https for
 {deb,security}.debian.org by default. With this bug report, I'll
 collect related things and fix it.

 - Update mirror list (how?)
 - Update security mirror setting in d-i (how?)
 - https://www.debian.org/mirror/list.en.html points http""://deb.debian.org,
   it should be https.
 - deb.debian.org says "deb http://;, it should be "deb https://;
 - In release notes, it should be https://security.debian.org, at least
   See 
https://www.debian.org/releases/stable/amd64/release-notes/ch-information.html#security-archive


*) https://lists.debian.org/debian-devel/2021/08/msg00269.html