Re: zstd compression for packages

[Steve Langasek]

On Sat, Mar 17, 2018 at 03:09:55PM +, Dimitri John Ledkov wrote:
> On 16 March 2018 at 22:13, Steve Langasek  wrote:
> > In other words: if we want to make this the default, we should quantify
> > Daniel's remark that he would prefer a 6% faster download over a 10% faster
> > unpack.

> Well, I think it does not make sense to think about this in absolute
> terms. Thinking about user stories is better.

Sure.

> A stable series user will be mostly upgrading packages from -security
> and -updates. The download speed and/or size of debs does not matter
> much in this case, as these are scheduled to be done in the background
> over the course of the day, via unattended upgrades download timer.
> Installation speed matters, as that is the window of time when the
> system is actually somewhat in a maintenance mode / degraded
> performance (apt is locked, there are CPU and disk-io loads).

Does unattended upgrades download both -security and -updates, or does it
only download -security?  From what I can see in
/usr/bin/unattended-upgrade, the allowed-origins check applies to both the
downloads and the installation.

So by default, increases in the download time of non-security SRUs would be
perceivable by the user (though perhaps not of interest).

> New instance initialization - e.g. spinning up a cloud instance, with
> cloud-init, and installing a bunch of things; deploying juju charm /
> conjure-up spell; configuring things with puppet / ansible / etc =>
> these are download & install heavy. However, users that do that
> heavily, will be in a corporate / bussiness / datacentre environment
> and thus it is reasonable to expect them to have either a fat internet
> pipe, and/or a local mirror. Meaning download speed & size, are not
> critical.

Generally agreed (but the assertion should still be tested, not assumed).

> Then there are devel series users, developers who do sbuild builds,
> etc. These users are most likely to be on slower home-user connections
> and watch things a lot more closely interactively, who indeed care
> about the total download+install time. These users, are most likely
> very vocal / visible, but are not ultimately the target audience as to
> why we develop Ubuntu in the first place. Thus I would be willing to
> trade personal developer/devel-series user experience, in favor of the
> stable series user. I'm not sure how much it makes sense to
> proxy/cache/local-mirror devel series, if it is only a single machine
> in use.

I disagree that we don't develop Ubuntu for developers.  The developer
desktop continues to be an important use case, and while it shouldn't
necessarily dominate every time there is tension between the desktop and
server use cases, it also shouldn't be ignored.

But furthermore, I think there's a separate use case you've not included
here, which is "client user selects a piece of software for installation and
wants to use it immediately".  In that case, the total clock time from
expression of intent, to when the package can be used, does matter.  And
it's not limited to developers of Ubuntu or people tracking the devel
series; this is relevant to the usability of the desktop in stable releases. 
It is also, I would argue, the use case that is most important in terms of
its impact on user satisfaction, because it's precisely in the critical path
of a task that has the user's attention; whereas improvements to the other
use cases may improve overall efficiency, but have little or no proximate
benefit to the human user.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: PGP signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Supporting LZ4 as initramfs compressor

[Steve Langasek]

Hi Balint,

On Mon, Mar 19, 2018 at 02:59:24PM +, Balint Reczey wrote:

> Initramfs-tools uses gzip compression by default which served us well
> for quite some time but LZ4 offers way faster decompression while
> making a only slightly bigger initramfs files.

When people have previously discussed lz4 on IRC as a possible choice for
default compression algorithm, I had the impression that this was with the
expectation that the resulting initramfs files would be *smaller* than with
using gzip.

(I think.  It happens that names for compression algorithms are remarkably
unoriginal, so it's possible I've confused lz4 with another.)

But your data shows that lz4-compressed initramfs is definitely larger than
gzip, which means that there are tradeoffs here that should be fully
examined.

After all, an initramfs that's not compressed at all would take even less
time to decompress at boot (0s) but would occupy more space.  But you aren't
advocating for this, so there must be some reason you believe lz4 is more of
a sweet spot than gzip?


The first thing that I see missing from this analysis is the time it takes
for the firmware/bootloader to load the initramfs into memory.  I know from
experience that some bootloader filesystem drivers have quite poor
performance; and the time spent loading the initramfs into memory will scale
roughly linearly with the file size.  So any analysis of lz4 impact on boot
speed needs to take this into account.  We should show that the overall
bootspeed from bootloader to pid 1 has actually improved, and this should be
measured with multiple bootloader driver implementations (across
architectures; UEFI vs BIOS; possibly on multiple virtualization substrates
vs. x86 bare metal).


The second thing to consider is that, regardless of any improvements in our
autoremoval of kernels, we have had some historical default sizing for /boot
partitions in the installer which are now on the small side for even a fully
correct kernel upgrade path.

In trusty, the default (max) size for a /boot partition was 256M.  In
xenial, it was 512M.  In artful, we have bumped this up to 768M with a
minimum of 512M because of LP: #1716999.

The actual space consumed by the static files in the 4.13.0-7-generic kernel
in artful - not counting the current .efi.signed duplication which will
hopefully go away soon - is just under 13MiB.  My bootloader here is 8MiB,
but 10MiB is not out of the question in some configurations.  My initramfs
is 52MiB rather than the 55MiB in that bug, but again 55MiB is plausible -
and your own numbers seem to show 56MiB.

That means that anyone who installed with trusty, has /boot as a separate
partition, and has plymouth in the initramfs (such as for encrypted root
disk, which would be a common reason to have a separate /boot) has already
run out of space on their /boot while using gzip; so must already reinstall
or switch to a non-default initrd compression algorithm on upgrade.  This
can therefore be ignored for the choice between gzip and lz4 as default.

Anyone who installed with xenial is borderline today with artful;
56*4+3*13+10 == 273MiB, which is more than xenial's minimum /boot size of
256MiB but less than the max of 512MiB.  So some number of users have a boot
partition that's large enough to accommodate gzipped initrd, but will run
out of space once you switch to lz4 (64*4+3*13+10 == 305MiB).  And those
that don't run out of space immediately as a result of the switch to lz4
would still run out of space sooner as the kernel size grows (since the
kernel definitely seems to only grow over time with new drivers).

Systems installed with artful or newer seem to still be fine for a while,
with either gzip or lz4.

So there's a decision to be made about whether we care about upgrades
working with the default compression on systems installed with smaller boot,
and for how long.  If we decide this shouldn't block switching the default
compression, we also need to sort out how we will communicate this to
affected users - and in particular, how to avoid problems on upgrade when
the user runs out of disk space at the worst possible time.


> Base on the results I plan adding LZ4 compression support to
> initramfs-tools as requested in LP: #1488620 [1] in the next days
> without setting it as default and I propose setting LZ4 as default for
> 18.10.

Since this is a non-default option and doesn't introduce any new
dependencies, this seems fine.  It also doesn't seem urgent to me; in terms
of the upgrade path, it doesn't need to be supported in 18.04 in order to
consider making it the default in 18.10.

Cheers,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: PGP signature
-- 
ubuntu-devel mailing 

Re: Samba CVE-2018-1057

[Thomas Ward]

It's already been patched.

The Ubuntu CVE tracker shows this [1], but also the relevant USN [2]
indicates that the issue is already 'fixed' in Ubuntu.  (It doesn't
always result in a software version bump, sometimes it's just patches
getting applied to 'fix' the issue in the given version of the package
in a specific Ubuntu release).


Thomas
LP: ~teward
Ubuntu Server Team Member


[1]:
https://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-1057.html
[2]: https://usn.ubuntu.com/3595-1/

(Oops forgot the links the first time)


On 03/20/2018 07:30 AM, James Boland wrote:
>
> Hi there,
>
>  
>
> Are there any plans to upgrade the current Samba package to mitigate
> again the recent security bug in CVE-2018-1057
>  ?
>
>  
>
> Cheers,
>
> James
>
>  
>
>
>

-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss

Re: Samba CVE-2018-1057

[Thomas Ward]

It's already been patched.

The Ubuntu CVE tracker shows this [1], but also the relevant USN [2]
indicates that the issue is already 'fixed' in Ubuntu.  (It doesn't
always result in a software version bump, sometimes it's just patches
getting applied to 'fix' the issue in the given version of the package
in a specific Ubuntu release).


Thomas
LP: ~teward
Ubuntu Server Team Member


On 03/20/2018 07:30 AM, James Boland wrote:
>
> Hi there,
>
>  
>
> Are there any plans to upgrade the current Samba package to mitigate
> again the recent security bug in CVE-2018-1057
>  ?
>
>  
>
> Cheers,
>
> James
>
>  
>
>
>

-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss

Re: Samba CVE-2018-1057

[Nish Aravamudan]

Hi James,

On Tue, Mar 20, 2018 at 4:30 AM, James Boland  wrote:
> Hi there,
>
>
>
> Are there any plans to upgrade the current Samba package to mitigate again
> the recent security bug in CVE-2018-1057 ?

https://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-1057.html

Thanks,
Nish

-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss

Samba CVE-2018-1057

[James Boland]

Hi there,

 

Are there any plans to upgrade the current Samba package to mitigate again
the recent security bug in CVE-2018-1057
  ?

 

Cheers,

James

 

-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss

Deprecation notice for mail-stack-delivery (dovecot) - planned to be dropped in 18.10

[Christian Ehrhardt]

Hello,
since Ubuntu 18.04 has reached feature freeze we started to think about
18.10.
One of the changes ahead is the dropping of the mail-stack-delivery package
(part of dovecot).

This package was created a long time ago with the intend to simplify
several steps of a mail server setup, for example to get a safe ssl secured
default installation.
It is essentially almost only a postinst to set up some better defaults.

But the bit that I can derive from bug reports and such indicates that it
is almost unused these days and has become an unused maintenance debt (it
started to show it's age - no more matching e.g. recommended ciphers).

Furthermore the world has moved on:
- For encryption ssl is now default in dovecot-core
- In general the hosting your own mail service has become less attractive
- If users want to set up a mail server still they often look more for e.g.
for Mail-in-a-box [1]

For all of these reasons we intend to drop the mail-stack-delivery package
in 18.10

If there is a big love/consumption of the package we might have missed
please speak up.
In that case we likely want to drop it from dovecot still, but community
could take over maintenance in a separate package that lives in universe.

[1]: https://mailinabox.email/


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Bugs reports should include syslog warnings or not?

[Matthew Paul Thomas]

Robie Basak wrote on 19/03/18 19:41:
>…> No, I think you have the inverse sense of what I intended. I mean
that> by the _developer_ choosing to write upstream code such that
something> is logged,
Ah, I see, I misinterpreted “one” as referring to the user.

>that developer is also implicitly deciding that the logs
> may be made public, because that's how the ecosystem works. So
> upstreams should ensure that private information is not logged by
> default.
>…
>> This seems to assume that the main use of Ubuntu log files is posting
>> in public bug reports and support forums — rather than, say,
>> troubleshooting and system administration in corporate IT
>> departments. Again, I’d be surprised if that’s true.
> 
> For a privacy concern, I don't think it matters what the main use is.
> A minority use that leads to a leak is still a leak that we should
> fix.

The proportion of use determines *how* it should be fixed. If many/most
uses of a log are for private troubleshooting and system administration,
then expecting every upstream developer to omit useful information when
logging — or to store “the private information somewhere
out-of-default-band” — would not be the most efficient solution.

-- 
mpt

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel