Re: Report a bug against which package - unattended-upgrades / apt / dpkg ??

2022-03-22 Thread Dan Ritter 
David Wright wrote: 
> On Mon 21 Mar 2022 at 15:07:45 (+), Dr. Alex Sheppard wrote:
> > On 21/03/2022 14:02, Dan Ritter wrote:
> > > Dr. Alex Sheppard wrote:
> > So, unless anyone can explain otherwise, I think there is a bug to
> > report against unattended-upgrades.
> 
> Perhaps. But I'd avoid the risks in autoremoving anything. In fact,
> I only automate the _downloading_ of upgradeable candidates.

That's how I do it for my household systems. At work, we
maintain mirrors and local repos and rank our systems so that a
problematic package change should show up first in a manual
test, then in a limited pool of machines, and hopefully never in
the production pools.

-dsr-

Re: Report a bug against which package - unattended-upgrades / apt / dpkg ??

2022-03-21 Thread David Wright 
On Mon 21 Mar 2022 at 15:07:45 (+), Dr. Alex Sheppard wrote:
> On 21/03/2022 14:02, Dan Ritter wrote:
> > Dr. Alex Sheppard wrote:
> > > 
> > >      Unattended upgrades ended up removing some of the packages it was was
> > > going to upgrade ... bind9 being one of them and thereby breaking DNS on a
> > > client's network.
> > > 
> > >      Is this a bug in unattended upgrades, or a bug in apt or dpkg? Here 
> > > is
> > > an extract from my unattended-upgrades.log to illustrate.

It might help to check other logs like dpkg.log¹, apt/history.log², and
auth.log³, and also the contents of /var/cache/apt/archives/{,partial/}⁴
to try to find out why bind9 wasn't upgraded. Was it even downloaded?
BTW does unattended upgrades send failure reports to /var/mail/sysadmin?

> > >      FTR: I'm struggling to think how bind9 could have been installed as a
> > > dependency for something else on the machine in question. I am pretty 
> > > sure I
> > > would have installed it manually which gives extra surprise to it being
> > > autoremoved.

Do you have backups of previous versions of /var/lib/apt/extended_states
which might give evidence of that. Otherwise, confirming a bug
might be difficult.

> > unattended-upgrades should not be allowed to autoremove. Doing
> > so always ends up with surprises, unless you have pre-tested
> > everything and keep your own apt repo a day or two behind
> > Debian's.
> > 
> > Unattended-Upgrade::Remove-Unused-Dependencies "false";
> > Unattended-Upgrade::Remove-New-Unused-Dependencies "false";

> My config had the normal "like apt-get autoremove" disabled as per default
> """
> // Do automatic removal of unused packages after the upgrade
> // (equivalent to apt-get autoremove)
> // Unattended-Upgrade::Remove-Unused-Dependencies "false";
> """
> 
> Whereas removing newly unused packages was enabled  - also as per the
> default
> """
> // Do automatic removal of newly unused dependencies after the upgrade
> // Unattended-Upgrade::Remove-New-Unused-Dependencies "true";
> """
> 
> My issue arose not from doing "the equivalent to apt-get autoremove"
> but by whatever "Remove-New-Unused-Dependencies" does. Does anybody
> know how the logic of this works / what commands are run to achieve
> this?

The logic would appear to be that Remove-Unused-Dependencies="true"
would effectively run an autoremove at some time in the process.
Assume package M0 depends on D0, and D0 was auto-installed. And say
that yesterday you removed M0, but ignored the fact that D0 could now
be autoremoved. You might expect that an unattended-upgrade would
remove D0 on account of Remove-Unused-Dependencies="true".

However, Remove-New-Unused-Dependencies="true" should leave D0 alone.

OTOH if a package M1 (having a dependency D1) was upgraded to a more
well-endowed version that dispensed with the services of D1, then
D1 now has the status of a newly unused dependency, and so
Remove-New-Unused-Dependencies="true" will autoremove it.

> The way I see it there is a bug in either:
> 
>     a) The logic in unattended-upgrades of how it goes about
> fulfilling "Remove-New-Unused-Dependencies"
> 
>         or
> 
>     b) An underlying command that is called to fulfill the
> "Remove-New-Unused-Dependencies" operation
> 
>         or
> 
>     c) Uninstalling a package that it just upgraded seems like a
> mistake that ought to be catchable, but if there is just no way to do
> the "Remove-New-Unused-Dependencies" operation without risk of
> something important getting [un]installed, the bug is that this is enabled
> by default.

I don't see why it is necessarily a mistake. In the example above,
there might be a package P1 that other people use, and which needs
D1 to be upgraded. It would then come down to a matter of the relative
timing of the decision to upgrade D1 and the decision to autoremove
it. I would say that the autoremove decision can only be taken /after/
the success (or failure) of all the upgrades involved.

> So, unless anyone can explain otherwise, I think there is a bug to
> report against unattended-upgrades.

Perhaps. But I'd avoid the risks in autoremoving anything. In fact,
I only automate the _downloading_ of upgradeable candidates.

¹ Did bind9 get upgraded at all?
² Did it intend to upgrade all the upgradeables?
³ Was the correct number of packages downloaded from debian-security?
⁴ Unfortunately, any recent apt-cleaning will prevent you knowing
  whether bind9…10u7 ever arrived on the system.

Cheers,
David.

Re: Report a bug against which package - unattended-upgrades / apt / dpkg ??

2022-03-21 Thread Dr. Alex Sheppard 


On 21/03/2022 14:02, Dan Ritter wrote:

Dr. Alex Sheppard wrote:

Hi,

     Unattended upgrades ended up removing some of the packages it was was
going to upgrade ... bind9 being one of them and thereby breaking DNS on a
client's network.

     Is this a bug in unattended upgrades, or a bug in apt or dpkg? Here is
an extract from my unattended-upgrades.log to illustrate.

     FTR: I'm struggling to think how bind9 could have been installed as a
dependency for something else on the machine in question. I am pretty sure I
would have installed it manually which gives extra surprise to it being
autoremoved.

unattended-upgrades should not be allowed to autoremove. Doing
so always ends up with surprises, unless you have pre-tested
everything and keep your own apt repo a day or two behind
Debian's.

Unattended-Upgrade::Remove-Unused-Dependencies "false";
Unattended-Upgrade::Remove-New-Unused-Dependencies "false";

However,
Unattended-Upgrade::Remove-Unused-Kernel-Packages
is usually safe, unless you have very specific reasons to keep
multiple old kernels around.

-dsr-

Thanks Dan, I've updated my config accordingly to avoid this happening 
in future :-)


It's not so urgent for me now, but I still get the feeling there is a 
bug to report.



My config had the normal "like apt-get autoremove" disabled as per default
"""
// Do automatic removal of unused packages after the upgrade
// (equivalent to apt-get autoremove)
// Unattended-Upgrade::Remove-Unused-Dependencies "false";
"""

Whereas removing newly unused packages was enabled  - also as per the 
default

"""
// Do automatic removal of newly unused dependencies after the upgrade
// Unattended-Upgrade::Remove-New-Unused-Dependencies "true";
"""


My issue arose not from doing "the equivalent to apt-get autoremove" but 
by whatever "Remove-New-Unused-Dependencies" does. Does anybody know how 
the logic of this works / what commands are run to achieve this?


The way I see it there is a bug in either:

    a) The logic in unattended-upgrades of how it goes about fulfilling 
"Remove-New-Unused-Dependencies"


        or

    b) An underlying command that is called to fulfill the 
"Remove-New-Unused-Dependencies" operation


        or

    c) Uninstalling a package that it just upgraded seems like a 
mistake that ought to be catchable, but if there is just no way to do 
the "Remove-New-Unused-Dependencies" operation without risk of something 
important getting installed, the bug is that this is enabled by default.



So, unless anyone can explain otherwise, I think there is a bug to 
report against unattended-upgrades.















Dr. Alex Sheppard
http://www.das-computer.co.uk

Re: Report a bug against which package - unattended-upgrades / apt / dpkg ??

2022-03-21 Thread Dan Ritter 
Dr. Alex Sheppard wrote: 
> Hi,
> 
>     Unattended upgrades ended up removing some of the packages it was was
> going to upgrade ... bind9 being one of them and thereby breaking DNS on a
> client's network.
> 
>     Is this a bug in unattended upgrades, or a bug in apt or dpkg? Here is
> an extract from my unattended-upgrades.log to illustrate.
> 
>     FTR: I'm struggling to think how bind9 could have been installed as a
> dependency for something else on the machine in question. I am pretty sure I
> would have installed it manually which gives extra surprise to it being
> autoremoved.

unattended-upgrades should not be allowed to autoremove. Doing
so always ends up with surprises, unless you have pre-tested
everything and keep your own apt repo a day or two behind
Debian's.

Unattended-Upgrade::Remove-Unused-Dependencies "false";
Unattended-Upgrade::Remove-New-Unused-Dependencies "false";

However, 
Unattended-Upgrade::Remove-Unused-Kernel-Packages
is usually safe, unless you have very specific reasons to keep
multiple old kernels around.

-dsr-