Re: [MBF] Moving mountnfs.sh script to runlevel 2
Dmitry Bogatov writes: > During resolving issue #551555 of bin:initscripts, I discovered that > solution would be move `mountnfs.sh' initscript into runlevels (2 3 4 5). > > Historically, `mountnfs.sh' was present in runlevel S due the fact, that > bin:initscripts were responsible to mounting /usr. Nowdays, /usr is > mounted by initramfs, so many scripts could be moved from runlevel S to > runlevels (2 3 4 5). > > Issue is that there is number of initscripts in runlevel S, that depends > on $remove_fs, which means /all/ file systems are mounted, not just / > and /usr. > > I ask corresponding maintainers (list at bottom of email) to either > * move their scripts to runlevel (2 3 4 5) [preferred] > * remove dependency on $remote_fs, if all they need is /usr > * replace with dependency on `mountall', if they need >one of (/run /dev/shm /tmp) What will happen on systems where users changed the configuration files and these changes are not applied automatically? Ansgar
Re: merged-/usr-via-symlinks vs a-different-layout
Guillem Jover writes:
> On Tue, 2019-02-19 at 08:54:12 +0100, Ansgar wrote:
>> Guillem Jover writes:
>> > 3) Switching packages to the merged-/usr layout could have been
>> >accomplished automatically via debhelper for a coverage of around
>> >99% (?) of the archive. With something along the lines of:
>> >
>> > ,---
>> > D=debian/tmp
>> > for d in /bin /sbin /lib; do
>> >for p in $(find $D/$d ! -type d); do
>> > b="${p##$D/}"
>> > m="$D/usr$b"
>> > if [ ! -e "$m" ]; then
>> >mkdir -p "$(dirname $m)"
>> >mv "$p" "$m"
>> >ln -s "${m##$D}" "$p"
>> > fi
>> >done
>> > done
>> > `---
>> >
>> >With the property that it would handle gracefully all cases were the
>> >maintainer has checked that no compat symlinks are required and has
>> >then progressively moved the pathname installation to their final
>> >destination under /usr.
>>
>> That is not merged-/usr, but a different filesystem layout.
>>
>> So, no, it is not possible to switch packages to merged-/usr this way.
>
> You are still conflating the concept with the deployment. The
> underlaying properties of merging /usr is that the contents for
> directories that are present in both / and /usr get merged into
> /usr.
No, I'm saying that you are proposing yet another different file system
layout. Which is quite simple to see: the file system would differ.
You just claim it follows similar "ideas" in some way.
I can come up with other variations that move files to /usr too: have
/bin a symlink to /usr/root/bin (some for other directories). That
would also be a different file system layout which shouldn't be called
"merged-/usr" either.
(That would have no filesystem aliasing between /bin and /usr/bin;
policy has special rules to allow replacing top-level directories with
symlinks; but besides that it is different for no good reason so I don't
think it would be a good idea.)
> See for example OpenSUSE which did this transition correctly:
>
> <https://en.opensuse.org/openSUSE:Usr_merge>
"The special comments #UsrMerge, #EndUsrMerge will help us to clean up
the spec files once everything has been moved and we can create a
general link from /bin to /usr/bin for example. "
So they want a /bin -> /usr/bin symlink (and the same for the other
directories) just as in the proposal you call broken...
Maybe you have misunderstood what SuSE plans to do?
Having dpkg track a symlink /bin/${something} and a file
/usr/bin/${something} will break should /bin be turned into a symlink
happen. So there is no nice way to finish the transition. I'm not sure
how that would work on RPM; maybe SuSE got stuck with that? That would
be another good reason to not follow that way...
I wouldn't be surprised if that took about as long as the /usr/doc ->
/usr/share/doc transition. (If the change process for most simple
changes like file locations takes 10+ years then I say the process is
broken...)
Ansgar
Re: merged-/usr-via-symlinks vs a-different-layout
Guillem Jover writes:
> 3) Switching packages to the merged-/usr layout could have been
>accomplished automatically via debhelper for a coverage of around
>99% (?) of the archive. With something along the lines of:
>
> ,---
> D=debian/tmp
> for d in /bin /sbin /lib; do
>for p in $(find $D/$d ! -type d); do
> b="${p##$D/}"
> m="$D/usr$b"
> if [ ! -e "$m" ]; then
>mkdir -p "$(dirname $m)"
>mv "$p" "$m"
>ln -s "${m##$D}" "$p"
> fi
>done
> done
> `---
>
>With the property that it would handle gracefully all cases were the
>maintainer has checked that no compat symlinks are required and has
>then progressively moved the pathname installation to their final
>destination under /usr.
That is not merged-/usr, but a different filesystem layout.
So, no, it is not possible to switch packages to merged-/usr this way.
> 4) Due to having to support the broken merged-/usr-via-symlinks
>deployment, when we want to move the contents of the binary
>packages to the merged-/usr layout, we require now to include tons
>of logic in (probably new) maintainer scripts, when we have been
>trying to remove them altogether. :( With even more files untracked
>by dpkg itself, bypassing the packaging system even more, when the
>compat symlinks could have been shipped in the binary packages.
As far as I know maintainer scripts are only required for moving files
from / to /usr when (a) a compat symlink is required, and (b) only when
both merged-/usr and non-merged-/usr is supported.
Ansgar
Re: Bug#877900: How to get 24-hour time on en_US.UTF-8 locale now?
Michael Stone writes: > On Thu, Feb 07, 2019 at 09:20:07PM +0100, Ondřej Surý wrote: >>en_DK.UTF-8 is a good default locale? > > I think the suggestion of just "en" made the most sense--specify the > language and an arbitrary set of rules that aren't tied to a specific > country. C.UTF-8 has the default of already existing and always being available. Other locales are not guaranteed to be around (well, except "C"). FWIW systemd will set LANG=C.UTF-8 if no other locale is specified since systemd 240: * When no /etc/locale.conf file exists (and hence no locale settings are in place), systemd will now use the "C.UTF-8" locale by default, and set LANG= to it. This locale is supported by various distributions including Fedora, with clear indications that upstream glibc is going to make it available too. This locale enables UTF-8 mode by default, which appears appropriate for 2018. That seems a reasonable choice and d-i could just use that by not specifying any locale if the user wishes so. (There is a small problem that getty@.service unsets LANG again.) (And you get 24-hour time, but very strange Endian in C.UTF-8: WEEKDAY MMM DD HH:MM:SS TZ while en_US.UTF-8 has at least DD MMM ... Having -MM-DD HH:MM:SS[+] instead would be much nicer if we were to create an arbitrary set of new rules for a new universal "en" locale ;-) ) Ansgar
Re: Removal of linux-base from jessie-backports broke Xen upstream CI
On Wed, 2019-02-13 at 12:46 +, Ian Jackson wrote:
> Alexander Wirt writes ("Re: Removal of linux-base from jessie-backports broke
> Xen upstream CI"):
> > Jessie backports doesn't exist anymore. For some time now. It is already on
> > its way to archive.d.o.
>
> Thanks are due to to folks on #debian-uk who pointed out this
> announcement (from June!) that I missed;
> https://lists.debian.org/debian-backports-announce/2018/07/msg0.html
>
> I have to say I'm not sure this removal is very helpful but then I'm
> not helping maintain -backports so I guess my opinion doesn't carry
> much weight.
More importantly Jessie has reached end-of-life[1]. Please do not
expect related suites (such as -security, -backports, -proposed-
updates, -updates) to continue working after this.
(Related to that: arm64 is gone from jessie-security; arm64 wasn't
removed from -backports as there is no LTS for backports and jessie-
backports will eventually be archived as is.)
Ansgar
[1] https://www.debian.org/News/2018/20180623
Re: Bug#922155: [Pkg-matrix-maintainers] ITP: matrix-archive-keyring -- OpenPGP archive key for the Matrix.org package repository
On Wed, 2019-02-13 at 15:41 +, Linda Lapinlampi wrote: > Template: matrix-archive-keyring/sources.list > Type: boolean > Default: false > _Description: Use APT data sources from Matrix.org? > The Matrix.org Debian package repository distributes supplemental Matrix.org > related packages intended to work with the Debian distribution, but require > software software outside of the distribution to either build or function. > These packages are digitally signed with keys from matrix-archive-keyring. > . > The Debian Project will be unable to directly support issues faced from using > supplemental packages from this third-party repository. Packages from these > APT sources may be non-conforming to the technical requirements set in the > Debian Policy for the Debian distribution. More important is the question if the system should /trust/ the keys. IMHO installing a non-Debian keyring should *not* make the keys trusted by APT by default (i.e. with the default answer if debconf is used). ubuntu-keyring does that; most other keyrings sadly do not follow this. Ansgar
Re: Use of the Build-Conflicts field
Sean Whitton writes: > Use of the Build-Conflicts field is currently mostly optional, but Ian > Jackson and I have been working on text for Debian Policy that would > require its use in certain cases. See #824495 for the discussion. > > There are two cases which we think that everyone would agree that there > is a bug, but we are not sure that the bug would be considered to be RC. > We are posting to -devel to see if, in fact, we do have a consensus that > these bugs would be RC, or not. > > (1) a package FTBFSs when: another package that is part of a "reasonable > standard development workstation install" is present, but the first > package does not declare a Build-Conflicts against the second > > (2) a package FTBFSs when: a package that is NOT part of a "reasonable > standard development workstation install" is present, but the first > package does not declare a Build-Conflicts against the second > > Is (1) an RC-severity bug in the package that FTBFSs? Is (2)? How many packages would be affected by (1) or (2)? I think both are less problematic than the case where the maintainer uploaded a package which has different features than a rebuild on a buildd. That would result in a rebuild (NMUs, next regular upload by someone else or a changed build environment, binNMUs) to change the features available to users. Something we really don't want; note that this could also happen in security or stable updates. (Having Build-Conflicts for the additional features case is probably not implementable with reasonable effort given how autotools and others enable automatic feature-detection by default which isn't really what one wants...) Failing to build in non-standard environments is in contrast a fairly friendly failure mode. So it should not be a serious bug (whether RC or not is something for the release team). > For the purposes of this e-mail, let's assume that we have a good grasp > on what a "reasonable standard development workstation install" means. I doubt we have, but let's ignore that. Ansgar
Re: Bug#877900: How to get 24-hour time on en_US.UTF-8 locale now?
On Thu, 2019-02-07 at 09:59 -0500, Michael Stone wrote: > On Thu, Feb 07, 2019 at 02:40:06PM +, Simon McVittie wrote: > > How would this locale differ from C.UTF-8? Is the only difference > > that C.UTF-8 has strict lexicographical sorting, whereas "en" would > > have > > case-insensitive sorting like en_GB.utf8 does? (If that's the only > > difference, then perhaps something like "LANG=C.utf8 > > LC_COLLATE=en_US.utf8" > > is enough.) > > POSIX specifies the output format for various utilities in the C locale, > which defeats my understanding of the purpose of this proposal. So, for > example, in ls -l: I don't think the "C.UTF-8" locale covered by any promises POSIX might make for "C". (Nor is what happens when no LC_*, LANG vairables are set at all.) Ansgar
Re: Potentially insecure Perl scripts
Russ Allbery writes: > Ben Hutchings writes: >> People have said this about ASLR, protected symlinks, and many other >> kinds of security hardening changes. We made them anyway and took the >> temporary pain for a long-term security gain. > > Well, Perl has a deprecation mechanism with warnings and so forth, > although I don't think Perl has ever actively broken a feature outside of > "use " with a later version, except for features marked as > experimental. But I suppose it's possible. '.' was eventually removed from @INC by default. It also wasn't seen as a security problem when I reported it as such (or not worth fixing at the time), but only years later when someone else reported it again. So maybe awareness changed a bit. But "<>" isn't the only problem, there are way too many uses of the two-argument form of Perl's "open" too... Ansgar
Re: package management symlink
Sören Reinecke writes: >> I'm not convinced that having to remember different package manager >> names is a significant problem for new people adopting a Linux >> distribution. The name is a very small part of the >> differences between >> the package managers (they have very different >> behaviour models, not >> just names), and the package managers are a small part of all the >> differences between distributions. > > Many developers don't provide their software via a repository instead > they provide the source code with an INSTALL file. They depend on > dependencies they resolve through the repositories. Approving "nimue" > just as symlink that points to the package management system across > linux systems would enable them to write just one install script for > debian and red hat systems for example. No, that already stops working when package are named differently which is frequently the case. There is no readline-devel package in Debian and no libreadline-dev in Red Had or Gentoo. Also what you suggest already exists, for example in the form of "pacapt" (but there are alternatives too!). What is the benefit of adding yet another version of these scripts? Ansgar
Re: [Idea] Debian User Repository? (Not simply mimicing AUR)
Paul Wise writes: > On Mon, Apr 8, 2019 at 3:34 PM Mo Zhou wrote: > >> However, the translator itself is not trivial, as it might need >> it's own shell parser or something alike to be reliable enough. > > Couldn't you just run makepkg (with some hooks) and dpkg-deb to > convert the results to Debian packages? How should this handle dependencies (probably named differently in Debian) or maintainer scripts? Tools like `alien` to convert RPM and Debian packages have similar limitations and I don't remember them working that well. Ansgar
Re: Hurd-i386 and kfreebsd-{i386,amd64} removal
Aurelien Jarno writes: > kfreebsd-amd64 and kfreebsd-i386 have now been moved to debian-ports. As > hurd-i386 has been moved earlier, it means that all the 3 architectures > have now been moved. I removed kfreebsd-* and hurd-i386 from ftp-master's unstable and experimental suites yesterday. The move should be completed with this. Ansgar
Re: ZFS in Buster
Ian Jackson writes: >> > Would it be possible to provide an alternative patched linux kernel >> > that works with ZFS? > > I think this would be both unwise legally (without seeking additional > legal advice) and rather rude to the kernel upstream whose code is > then being reused without permission - indeed, contrary to their > explicitly stated intent. At least one commercial distribution (Ubuntu) has been distributing ZFS binary modules as part of their Linux packages for some years and didn't have any problems. Debian doesn't provide prebuilt binary modules for out-of-tree modules mostly to not have to care about them when the Linux ABI changes as happens in many updates (including security/stable updates). (Ubuntu avoids that by using `wget` to get the source for out-of-tree modules like ZFS when building their src:linux package, something Debian would probably not like that much...) Ansgar
Re: Why do we take so long to realise good ideas (Was: Difficult Packaging Practices)
On Wed, 2019-05-29 at 10:38 +0200, Raphael Hertzog wrote:
> On Wed, 29 May 2019, Andrey Rahmatullin wrote:
> > One of the popular answers to this and some other problems is "nobody sat
> > down and wrote the code". Not sure what can we do about this class of
> > reasons.
>
> Use the $300,000 on our bank accounts?
I heard that this didn't work out well the last time ("dunc tank"),
though that was before the time I followed Debian development.
Ansgar
getting rid of "testing"
Hi,
what do people think about getting rid of current suite names ("stable",
"testing", "unstable") for most purposes? We already recommend using
codenames instead as those don't change their meaning when a new release
happens.
Related to that I would like to be able to write something like
deb http://deb.debian.org/debian debian11 main
deb http://security.debian.org/debian-security debian11-security main
in sources.list as codenames confuse people.
Ubuntu already has no suite names, only codenames, but having to map
"Ubuntu 18.04" to "bionic" instead of just writing the version in
sources.list is annoying (I always have to look up the codename to be
sure as I don't use Ubuntu that much).
Ansgar
Re: getting rid of "testing"
On Tue, 2019-06-25 at 16:39 +0800, Paul Wise wrote:
> On Tue, Jun 25, 2019 at 2:08 PM Ansgar wrote:
> > what do people think about getting rid of current suite names ("stable",
> > "testing", "unstable") for most purposes? We already recommend using
> > codenames instead as those don't change their meaning when a new release
> > happens.
>
> I use these (testing, etc) so getting rid of them would be annoying.
The "stable" suite names are more annoying than
unstable/testing/experimental as they require updates to suites at
release time that are not related to the release. That shouldn't be
necessary.
For "testing", "unstable" one could probably introduce some `Alias`
field in Release, but I also like minimalist solutions (which already
seem to work well for Ubuntu).
> > Related to that I would like to be able to write something like
> >
> > deb http://deb.debian.org/debian debian11 main
>
> Already kind of possible:
>
> deb http://deb.debian.org/debian Debian9.9 main
Yes, but it gives warnings for issues that I believe should be an error
instead. (And currently a good reason for TLS to talk to mirros so a
MitM that is not a mirror operator cannot give you oldstable when you
want to use unstable.)
debootstrap gives an error for this:
+---
| $ /sbin/debootstrap --print-debs Debian9.9 . http://ftp.de.debian.org/debian
unstable
| [...]
| E: Asked to install suite Debian9.9, but got stable (codename: stretch) from
mirror
+---
As Adam already pointed out having the point release in there also
makes "Debian9.9" rather unhelpful.
> > Ubuntu already has no suite names, only codenames, but having to map
> > "Ubuntu 18.04" to "bionic" instead of just writing the version in
> > sources.list is annoying (I always have to look up the codename to be
> > sure as I don't use Ubuntu that much).
>
> They do have the 'devel' suite, but it is not a proper one:
>
> https://bugs.launchpad.net/ubuntu/+bug/1821272
That is what Debian9.9 (and similar) are currently as well.
Ansgar
Re: Content Rating System in Debian
Hi, On Tue, 2019-06-25 at 11:40 +0700, Bagas Sanjaya wrote: > Based on above, what are your opinions/thoughts/positions about > Content Rating System in Debian? is this related to your other proposal involving giving "sudo" permissions to teenagers to handle this age recommendation stuff for TV programs? | In fact, many television stations have most programs written for | teens (age 13 and older), so sysadmins there configure sudoers which | allows teens to behave like sysadmins themselves (by giving them full | administrator privileges) on their production systems. Also, parental | monitoring and guidance can reduce likehood of teens breaking such | systems. Maybe because teens are largest marketshare for TVs. Ansgar - rating "kill -KILL" X-rated for extreme violence
Re: .deb format: let's use 0.939, zstd, drop bzip2
On Thu, 2019-05-09 at 12:32 +, Jeremy Stanley wrote: > On 2019-05-09 06:27:36 +0900 (+0900), Mike Hommey wrote: > > On Wed, May 08, 2019 at 09:04:49PM +, Jeremy Stanley wrote: > [...] > > > Are you talking about source packages or binary packages here? > > > The > > > latter use ar, not tar. > > > > Binary packages use both. > > > > $ ar t /var/cache/apt/archives/libgcc-9-dev_9.1.0-1_amd64.deb > > debian-binary > > control.tar.xz > > data.tar.xz > > Ahh, yes, thanks I should have remembered that... so then my > question becomes: is the suggestion to replace *both* ar and tar in > the binary package format with a single flat archive, or to switch > out the tarballs inside the ar archive but continue using ar to > aggregate them? `ar` needs to be replaced for the file size limitation mentioned in the initial mail: ar represents file size as a 10 digit decimal number[1] which limits the members (control.tar.*, data.tar.*) to ~10G. [1] https://en.wikipedia.org/wiki/Ar_(Unix)#File_header Replacing `ar` is an incompatible format change. So if we already do an incompatible change, it is an appropriate time to bundle any other incompatible changes (if there are any). That is why I suggested that it might be useful to also replace the `tar` archives with another format. Ansgar
Re: .deb format: let's use 0.939, zstd, drop bzip2
Adam Borowski writes: > On Wed, May 08, 2019 at 10:35:58PM +0200, Ansgar wrote: >> Adam Borowski writes: >> > I've recently did some research on how can we improve the speed of >> > unpacking >> > packages. There's a lot of other stages that can be improved, but let's >> > talk about the .deb format. >> > >> > First, the 0.939 format, as described in "man deb-old". While still being >> > accepted by dpkg, it had been superseded before even the very first stable >> > release. Why? It has at least two upsides over 2.0: >> >> Switching to a different binary format will break various tools. > > The 0.939 format is already supported by most tools. > > No one sane digs into insides of the format, using a small number of > low-level tools, thus we can reuse it with little effort. > > Of course, adding a new compressor _does_ break compat, but we added four > compressors to 2.0 over the years already, and the sky didn't fall. Well, it causes minor breakage which is fairly easy to fix. A different container format instead of tar would require more involved changes in tools, so I'm not 100% convinced of my idea myself ;-) The thread just looked like the right time to consider such changes. >> If we want to do this, I wonder if we shouldn't take the chance to move >> away from tar? > > Any seekable format significantly reduces compression, although this can > be reduced by managing split points. Well, depending on how much splitting you do, the loss in compression should be small enough to not care about? >> We have various applications that only want to extract single members of >> the package (changelog, NEWS, copyright, ...); tar is a really bad >> format for such an operation. Other formats (zip, 7z, ...) are more >> suited for them. > > Perhaps such files could be considered metadata and moved to the control > tarball? Or merely just moved forward -- remember that tarballs are > unordered. I don't think that is a good idea: if someone wants to use another file in a similar way, he couldn't and would have to fall back to the worse solution. As an example: I have a config-diff script which compares conffiles with the pristine version included in the *.deb; it wants to access /etc/*. (Though ideally dpkg would keep the pristine version accessible below /usr; that would also be useful for other uses.) Also dpkg keeps metadata in /var, but changelogs, NEWS, copyright documentation isn't variable state data and should be below /usr... The same is really true for lists of files and maintainer scripts though. Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Thu, 2019-05-09 at 10:33 +0100, Ian Campbell wrote:
> On Thu, 2019-05-09 at 11:19 +1000, Ben Finney wrote:
> > What I remain unconvinced of is the worth of abandoning the clean
> > separation between an upstream source repository (whether Git repository
> > or not) and a VCS repository for Debian packaging (typically in Git).
>
> I think there's a common misconception here which has cropped up
> several times in this thread. (NB: I've not used dgit in anger yet, but
> I hope what follows is correct).
>
> That misconception is that "dgit repo" == "maintainer's repo", which is
> not actually the case. As a maintainer you can (if you want) just use
> "dgit push" (instead of dput) while only caring about (and interacting
> with) the "maintainer's repo", never touching or looking at dgit's view
> of the world (apart from perhaps noticing and ignoring some `dgit/*`
> branches in your _local_ repo, I don't beleive you are required to push
> those to anywhere, including your maintainer repo).
I think that is very confusing, yes.
By now I have come to understand that whatever "dgit" produces should
be just regarded as a build artifact, just like binary packages or for
some people the .dsc or for some workflows debian/patches/*.
Though directing users to build artifacts when they request the source
seems counterproductive to me...
I wonder why not the "maintainer view" is published (for the part
making Git repositories "reliably locatable") and any mangling is
applied by "dgit clone" (instead of "dgit push")? ("dgit push" can
check that the mangling works.)
Then whatever is published is what is the actual preferred form of
modification (as it is what the maintainer uses), but if so desired one
can still get a "dgit view". (Though for contributing changes to the
maintainer, one should probably base them on the maintainer view...)
In this case the published history also matches the "git histories we
are actually using ourselves", a design goal not met currently; one
could also apply the mangling feature to repositories not published on
the dgit server.
Ansgar
Re: .deb format: let's use 0.939, zstd, drop bzip2
On Fri, 2019-05-10 at 11:04 +0200, Thomas Goirand wrote:
> On 5/9/19 6:25 PM, Andrej Shadura wrote:
> > How about the format opkg used for some time, which is a .deb file
> > but
> > with tar as the outer container format instead of ar?
>
> This is a very bad idea. When installing a large amount of packages,
> apt
> needs to uncompress all control.tar.gz files so it can get the config
> and templates of debconf. With tar, meaning without an index, one may
> need to uncompress the whole of the .deb file in order to extract
> just a
> tiny portion of it. This could potentially be super long (think: a
> dist-upgrade with so many packages...).
The outer container would probably be uncompressed; as it has only very
few members (dummy file, control.tar.*, data.tar.*, ${something-new}),
accessing any of these few members is fast (as you can just seek from
one header to the next and only need to do so few times).
Ansgar
Re: .deb format: let's use 0.939, zstd, drop bzip2
Hi Adam, On Fri, 2019-05-10 at 16:11 +0200, Adam Borowski wrote: > /usr on the box I'm sitting at: > * zip the program: dies horribly due to /usr/lib/llvm-7/build/ > symlink > loops. > * zip: > 1891345142 bytes > * zip-the-concept (individually compressed files), xz > 1516943024 bytes > * tar.xz > 1092591508 bytes > > Linux source: > * zip: > 213820843 bytes > * individually compressed files, xz > 180997203 bytes > * tar.xz: > 104318396 bytes > > So no, I don't want zip, nor even a randomly accessible format. Could you also try 7z? It supports solid compression[1] which compresses multiple files into one block like tar.xz, but unlike tar.xz can use more than one block: "Later versions of 7-zip use a variable solid block size, so that only a limited amount of data must be processed in order to extract one file". So it would provide somewhat random access and still offer better compression than zip or other forms of individually compressed files. It would be interesting if you could also include the access time for extracting only the last member of the archive; though for 7z one would need to check if it does the right thing first... Ansgar [1] https://en.wikipedia.org/wiki/Solid_compression
Re: .deb format: let's use 0.939, zstd, drop bzip2
Adam Borowski writes: > On Mon, May 13, 2019 at 11:25:11AM +0200, Ansgar wrote: >> It supports solid compression[1] which >> compresses multiple files into one block like tar.xz, but unlike tar.xz >> can use more than one block: "Later versions of 7-zip use a variable >> solid block size, so that only a limited amount of data must be >> processed in order to extract one file". > > Yeah but the default block size is 2GB, which would make every single .deb > in the archive effectively fully solid. You can fine-tune the values, but > example numbers don't appear like they'd help much. > > And just tested, -ms=on vs -ms=16m: > -rw-r--r-- 1 kilobyte kilobyte 1087957645 May 13 14:37 u.7z > -rw-r--r-- 1 kilobyte kilobyte 1253369019 May 13 14:50 u16m.7z > > So the space loss is massive. That indeed looks like it likely not worth switching to a different format to allow better random access (which is more of a special-use case anyway). Thank you for your experiments! Ansgar
Re: .deb format: let's use 0.939, zstd, drop bzip2
Jeremy Stanley writes: > On 2019-05-08 22:35:58 +0200 (+0200), Ansgar wrote: >> Switching to a different binary format will break various tools. If we >> want to do this, I wonder if we shouldn't take the chance to move away >> from tar? >> >> We have various applications that only want to extract single members of >> the package (changelog, NEWS, copyright, ...); tar is a really bad >> format for such an operation. Other formats (zip, 7z, ...) are more >> suited for them. > > Are you talking about source packages or binary packages here? The > latter use ar, not tar. I'm talking about binary packages (*.deb). They currently use tar archives (control.tar.*, data.tar.*) packed in an ar archive. Ansgar
Re: .deb format: let's use 0.939, zstd, drop bzip2
Adam Borowski writes: > I've recently did some research on how can we improve the speed of unpacking > packages. There's a lot of other stages that can be improved, but let's > talk about the .deb format. > > First, the 0.939 format, as described in "man deb-old". While still being > accepted by dpkg, it had been superseded before even the very first stable > release. Why? It has at least two upsides over 2.0: Switching to a different binary format will break various tools. If we want to do this, I wonder if we shouldn't take the chance to move away from tar? We have various applications that only want to extract single members of the package (changelog, NEWS, copyright, ...); tar is a really bad format for such an operation. Other formats (zip, 7z, ...) are more suited for them. Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
Russell Stuart writes: > On Tue, 2019-04-30 at 09:25 +0800, Paul Wise wrote: >> I like this option because it still works well if we ever decide to >> fix a fundamental flaw in the Debian source package layout. > > I suspect whether that's a fundamentally is a matter or personal taste. > On this point my taste aligns with yours. > > I've used both rpm source format and the Debian one, and IMO the rpm > one is mostly better. The primary reason is the one you've mention > here: they maintain the separation between the source, rpm spec, and > build areas's far more cleanly than Debian does. This makes some > common flaws one often flaws in Debian packages just disappear: like > cleaning up the source directory after a build. I also agree with this. It is also not only RPM, but for example also Gentoo, Arch and others which contain only the distro-specific parts in their repositories. This also makes things easier as developers do not have to know about branch management, merging, rebasing, ... to start with. Many people using Git don't know how to do this. As an example: to update to a new upstream release, I ideally just have to drop the new upstream tarball, update d/changelog and am done. Compare with [1] which is much more complicated, even ignoring the extra complexity using dgit adds compared to just using git. Ansgar [1] https://manpages.debian.org/stretch-backports/dgit/dgit-maint-merge.7.en.html#NEW_UPSTREAM_RELEASES
Re: Preferred git branch structure when upstream moves from tarballs to git
Sam Hartman writes: > I'm having a bit of trouble here and am hoping you can help us out. > Ian asked what git workflow it is that you're talking about where people > can deal with commit push and pull and don't need to know more. > > I didn't see a clear answer to that. > Which debian packaging workflow do you believe has that property? There is no need for a vendor branch if only the packaging information is kept in the repository, i.e. only debian/. In particular there is no need for branches or merges for regular updates (i.e. not based on an older release). This is also what pretty much every other distribution seems to be doing, for example Fedora, Gentoo, Ports (BSD), Arch Linux, Void Linux. It would be easier if there was even more separation between packaging and upstream source, e.g. base/src (upstream source), base/debian (packaging information), and possibly other directories below base for build artifacts (instead of unpredictable locations under base/debian). Which leads back to the beginning of the subthread[1]. [1] https://lists.debian.org/debian-devel/2019/04/msg00462.html Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Tue, 2019-05-07 at 12:51 +0100, Ian Jackson wrote:
> Ansgar Burchardt writes ("Re: Preferred git branch structure when
> upstream moves from tarballs to git"):
> > Sam Hartman writes:
> > > OK, I didn't hear that as an answer but think I'm coming to the
> > > same
> > > conclusion that Ian did after reading your mail.
> > > It sounds like you are talking about having the Debian packaging
> > > in a
> > > separate repository from upstream.
> > > Do I correctly understand the model you are talking about?
> ...
> > I'll point to [1] for what complexity this avoids. Try explaining
> > that
> > to someone without advanced Git knowledge...
> >
> > [1]
> > https://manpages.debian.org/unstable/git-debrebase/git-debrebase.5.en.html
[...]
> In this message you are using the fact that I have written a
> comprehensive data model specification, against me. This is a
> seriously bad thing to do.
>
I think you underestimate how much knowledge you require from users
right from the beginning...
So in a way the problem is that the documentation needs to exist. git-
debrebase or dgit (to a lesser extend) implement a Debian-specific
version control system on top of Git. git-debrebase makes this most
obvious: the Git history it generates is pretty much unlike anything
you would ever get from regular Git usage (counterexamples welcome), it
mangles your Git commits into a specific format it expects
(laundering), mangles rebased branches into fast-forward merges with
pseudomerges; all this does not happen in regular Git usage.
dgit has less of these problems, but still does things different from
regular git usage (again pseudomerges to make rebases fast-forward for
example); there are other issues as well such as its suggested use for
NMUs feeling rather unfriendly as it moves the packaging to another VCS
than the maintainer uses with a disjoint history...
Sure, dealing with patch series is not nice either way, but I don't
think the added complexity of git-debrebase (or dgit) is worth what it
provides. Alternatives require less intimiate knowledge of git and
provide a gentler learning curve; some basic patch queue manipulation
even doesn't require special tools at all (adding/removing patches that
already exist in the right format and apply).
Other distributions have intentionally be trying to move to less
distro-specific tools to reduce barrier of entry; dgit and git-
debrebase both move in the opposite direction (more highly distro-
specific tools).
> You should be applauding me for providing serious documentation for
> advanced users.
According to git-debrebase(1) the documentation I referred to is basic
documentation:
> You should read this manpage in cojnunction with "TERMINOLOGY" in
> git-debrebase(5), which defines many important terms used here.
"TERMINOLOGY" then refers to most other sections.
> The obvious counter is the (still comprehensive, but user-facing)
> tutorial manpage. For example,
>
> https://manpages.debian.org/stretch-backports/dgit/dgit-maint-debrebase.7.en.html#EDITING_THE_DEBIAN_PACKAGING
> et seq.
It illustrates my other main gripe with dgit/git-debrebase: it makes it
harder to share in-progress work, in fact git-debrebase discourages
people from doing so:
| Note that each time you conclude a debrebase you introduce a
| pseudomerge into your git history, which may make it harder to read.
| A simple convention you can use to minimise the number of
| pseudomerges is to git debrebase conclude only right before you
| upload or push to salsa.debian.org.
Yes, one can avoid some of the problems by pushing a non-fast-forward,
non-interchange branch. But that differs from the regular workflow
and, again, requires more advanced Git knowledge.
> An equivalently detailed and frank document about dpkg-source would
> be a nightmare.
The concept behind 3.0 (quilt) is much easier to explain: extract
tarballs, optionally apply some patches provided. With the bonus that
users who have used other distributions before might already know about
most of this which lowers barrier of entry. Implementation details
like constructing a .dsc are indeed messy.
With most other Debian packaging workflows one can also limit oneself
to more basic Git commands for regular usage, in the most minimalistic
case only commit/push/pull (not even merge); these also match what
users will know if they have used Git at all (be it for packaging in
other distribution, upstream projects, or even just writing their
thesis).
You might say that git-debrebase is not required, but what about users
that want to contribute to a package maintained using it? They will
have to deal with the complexity... This also differs from other
packaging workflows that usually don't make specific tools mandatory.
(Not all tools can always be made optional of course, but one should do
some consideration if one makes new tools mandatory.)
Ansgar
Re: Discussion on eventual transition away from source packages
Daniel Kahn Gillmor writes: > On Fri 2019-03-22 09:32:55 +0100, Lucas Nussbaum wrote: >> I'm probably missing something, but it doesn't sound like a lot of work >> to me? It's "just" a service that: >> - gets notified of the existence of a git repo + tag to upload >> - fetches that git repo + tag >> - checks signature / confirm that the GPG key owner is allowed to upload >> that package > > In case anyone is considering trying to do this, please be aware that > there are several non-obvious subtleties involved in "verifying a git > tag". Doesn't Git also only use hash algorithms that are no longer recommended for cryptographic applications? Or have they finished moving to stronger algorithms? I don't think we should downgrade to SHA-1 for new services. Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Thu, 2019-05-02 at 13:45 +0100, Ian Jackson wrote:
> Ansgar Burchardt writes ("Re: Preferred git branch structure when
> upstream moves from tarballs to git"):
> > On Tue, 2019-04-30 at 16:00 -0700, Sean Whitton wrote:
> > > As a package maintainer, if you don't keep the whole source package in
> > > git, you're giving up on a lot of the power of git.
> >
> > I think keeping entry barriers low is more important than being able to
> > use all the power of Git. That's sadly one of the main problems with
> > Dgit: it raises entry barriers by making packaging more complicated.
>
> A source-only upload with dgit is fewer commands, and more reliable,
> than doing so with sbuild/dput.
Complexity is not the number of commands to use.
Having to know about branches, merging, dealing with multiple remotes,
... *is* an entry barrier compared to not having to know about it. Now
you have to teach people that before you even get to how to write a
build recipe.
(Also for source-only uploads you don't need sbuild at all.)
> Doing an NMU campaign with dgit is massively easier than doing so with
> .dsc-based tools.
Why should that be "massively easier" with dgit? Without dgit you get
source, change the source package, build the source package, upload the
modified source package. No matter what workflow/VCS/* the maintainer
uses.
Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Thu, 2019-05-02 at 09:15 -0700, Russ Allbery wrote:
> Ansgar writes:
>
> > Having to know about branches, merging, dealing with multiple remotes,
> > ... *is* an entry barrier compared to not having to know about it. Now
> > you have to teach people that before you even get to how to write a
> > build recipe.
>
> I'm confused. I'm a happy user of dgit and don't have to think about any
> of those things as part of using dgit. I choose to use branches, but I
> certainly wouldn't have to, and merging, multiple remotes, and so forth
> don't seem related to using dgit at all.
How do you update the package to a new upstream release?
The "dgit" repository is also separate from the "real" repository; if
you just use "dgit clone ${something}" you won't get the current
"master" branch (unless it happens to be identical to the last
release), or totally different if the maintainer doesn't use dgit.
The history is also strange if you "dgit clone" a repository where the
maintainer used dgit in the past, but no longer does. Now you have a
commit tree with multiple roots which is also confusing for people.
All of this is very uncommon outside the dgit world.
Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Fri, 2019-05-03 at 15:59 +0100, Ian Jackson wrote:
> Ansgar writes ("Re: Preferred git branch structure when upstream moves from
> tarballs to git"):
> > On Thu, 2019-05-02 at 09:15 -0700, Russ Allbery wrote:
> > > Ansgar writes:
> > > > Having to know about branches, merging, dealing with multiple remotes,
> > > > ... *is* an entry barrier compared to not having to know about it. Now
> > > > you have to teach people that before you even get to how to write a
> > > > build recipe.
> > >
> > > I'm confused. I'm a happy user of dgit and don't have to think about any
> > > of those things as part of using dgit. I choose to use branches, but I
> > > certainly wouldn't have to, and merging, multiple remotes, and so forth
> > > don't seem related to using dgit at all.
> >
> > How do you update the package to a new upstream release?
>
> The same way you would with whatever tools you are currently using for
> that task. dgit is not a git delta management tool. For the
> maintainer, dgit replaces dput, not gbp pq or git-dpm or quilt.
The question was to illustrate that dgit forces you to learn about
branches, merging, ... There is no workflow with dgit that doesn't
require this; when not using dgit, this isn't required.
Using dgit is a significantly higher entry barrier than not using it; I
wouldn't recommend using it for that reason.
> The "dgit" repository is also separate from the "real" repository;
> > if
> > you just use "dgit clone ${something}" you won't get the current
> > "master" branch (unless it happens to be identical to the last
> > release), or totally different if the maintainer doesn't use dgit.
> >
> > The history is also strange if you "dgit clone" a repository where
> > the
> > maintainer used dgit in the past, but no longer does. Now you have
> > a
> > commit tree with multiple roots which is also confusing for people.
> >
> > All of this is very uncommon outside the dgit world.
>
> If you are trying to get the source code, there is a choice to be
> made.
>
> You can choose to
>
> (1) Get a predictable standardised tree format, which can directly be
> built, cherry-picked, etc.,
>
> And, always have what is actually in the archive.
>
> But maybe an unhelpful history. Especially, if the maintainer
> does not use dgit push.
>
> (dgit clone)
This doesn't give you the real source according to people who state
that "source code" nowadays means the Git repository used by the
maintainer. dgit doesn't differ from apt-get source & gbp import-dsc
there, at least I don't see one.
(This might be different for packages where the maintainer is using
dgit, didn't forget to push and didn't make any changes since the last
upload, i.e. when debcheckout would give the same result.)
> (2) Have an unpredictable (and usually not even specified) tree
> format, which may (and usually will) require use of
> git-workflow-specific tools to even build it.
>
> Have to understand the maintainer's branch structure to try to
> figure out which version corresponds to what is in the archive.
>
> Have the maintainer's history. This is usually in a format
> useful to the maintainer, but it is not standardised. It may be
> the packaging history for a whole packaging ecosystem. It may be
> a linear history of a gbp pq branch. It may be a git-dpm history
> containing multiple rebases of the same patch series.
>
> (debcheckout)
It has the advantage of having the history if one cares about it. Why
would one not prefer that over the result of importing the .dsc
somewhere?
Maybe dgit is useful for some people to maintain their packages in Git;
but it's not helpful for packages where the maintainer is using it and
some complications are just not a good idea IMHO (like using a separate
Git repository with potentially different contents and not warning
users when fabricating history different from the real VCS history).
Ansgar
Re: Preferred git branch structure when upstream moves from tarballs to git
On Fri, 2019-05-03 at 17:39 +0100, Ian Jackson wrote:
> Ansgar writes ("Re: Preferred git branch structure when upstream
> moves from tarballs to git"):
> > On Fri, 2019-05-03 at 15:59 +0100, Ian Jackson wrote:
> > > Ansgar writes ("Re: Preferred git branch structure when upstream
> > > moves from tarballs to git"):
> > > > How do you update the package to a new upstream release?
> > >
> > > The same way you would with whatever tools you are currently
> > > using for
> > > that task. dgit is not a git delta management tool. For the
> > > maintainer, dgit replaces dput, not gbp pq or git-dpm or quilt.
> >
> > The question was to illustrate that dgit forces you to learn about
> > branches, merging, ... There is no workflow with dgit that doesn't
> > require this; when not using dgit, this isn't required.
>
> This is so wrong it is very perplexing to me.
Yet you confirm exactly what I am saying...
> This does not work right now with dgit because I have not taught dgit
> how to convert such a thing into a git tree that contains the actual
> source code.
I'm not talking about a hypothetical, different version of dgit which
might have a different set of features. Anything is possible for such
a hypothetical version.
> Using dgit is a significantly higher entry barrier than not using
> > it;
>
> Using dgit is a much lower entry barrier if you already know git, and
> don't know about Debian source packages. Like most people nowadays.
I know plenty of people who more or less only know git commit, push and
pull. Nothing more advanced.
> Using dgit is only a higher entry barrier if (i) you aren't
> comfortable with "intermediate" use of git (ii) you are already an
> expert on dpkg-source, quilt, etc.
Thank you for confirming that dgit raises entry barriers. I still
believe that is opposite of what distributions should do, see the
popularity of AUR or similar projects.
> Any maintainer who feels (as I often do) that the preferred form for
> modification of their package includes the full git history, should
> already be using dgit, since how else can they always discoverably
> publish the complete corresponding source ?
There is this fancy Vcs-Git field. Simple to discover.
Now, if dgit could just use that repository...
> [1] Possibly the user will get the maintainer's git history plus some
> autogenerated dgit conversion commits. For example, for a package
> whose maintainer uses gbp pq, the user gets a thing that has many
> similarities to the gbp pq patch queue branch. In particular, the
> patches are applied, each as a git commit in the user's history.
And then has to take care to not push those artificial commits
somewhere if that is not the preferred workflow... That seems rather
backwards to me.
Ansgar
Re: git & Debian packaging sprint report
Sean Whitton writes: > On Fri 12 Jul 2019 at 02:06PM +02, Ansgar wrote: >> Depends on a lot of things. As far as I understand this work is in a >> very early stage and a first brainstorming session on what problem this >> is intended to solve, why one should consider doing it, and possible >> requirements is planned for DebConf[1]. >> >> Without having any of these, it is hard to comment on anything. > > Figuring out how this is going to be deployed as Debian infrastructure > is indeed at an early stage. > > I don't think it is accurate to think of the whole project as being at > an early stage. From my perspective, the hardest problem to solve was > conversion of git trees to uploads, on the server side, in a way that is > user friendly. We take ourselves to have solved that problem, which to > my mind renders this project beyond the early stages of development. What is the DebConf BOF then for? It says it is "a requirements collecting BOF for that [git push] discussion"? Ansgar
Re: Bypassing the 2/3/4GB virtual memory space on 32-bit ports
"Theodore Y. Ts'o" writes: > On Wed, Aug 21, 2019 at 10:03:01AM +0100, Luke Kenneth Casson Leighton wrote: >> so i hope that list gives a bit more context as to how serious the >> consequences of dropping 32 bit support really is. > > I very much doubt we are any where near "dropping 32-bit support". > There's a lot of "all or nothing thinking" in your argumentation > style. The topic of this very thread is how to avoid dropping support for 32bit architectures. So people clearly want to continue supporting it. > As Sam has said multiple times, what's much more likely is that the > set of packages that can't be built on native packages on 32-bit > platforms will grow over time. The question is when will that > actually *matter*? There are many of these packages which no one > would want to run on a 32-bit platform, especially if we're focusing > on the embedded use case. It might matter pretty soon: for generic-purpose systems one almost always want to be able to use a web browser; even various embedded systems do. However web browsers are one of the most heavy packages to build. > At least initially, the simplest way of dealing with the problem is > that maintainers will simply not build their package on 32-bit > platforms. I don't think we want this to happen if we want to continue supporting 32bit systems. But using a 64bit toolchain in an otherwise 32bit userland as the thread suggests we now really should start looking at will avoid this. It will probably take a few more years until we have to worry about being able to run browsers in 32bit... But I would hope that most people understand that using 32bit for new generic-purpose systems is not future-proof (and hasn't been for a while); even phones have started to move to 64bit systems a while ago. Ansgar
Re: Git Packaging: Native source formats
Sam Hartman writes: > * One is that you're not using upstream tarballs. If upstream has > tarballs they produce, we're not using them. I guess we may end up > having that part of the conversation now rather than later. > > It's clear that we value integrity of upstream source. That is we > want to make it easy for people to start from some upstream source > that is trusted because upstream has produced it and audit just our > changes. > One way to do this is with an upstream tarball and a diff (or set of > diffs or a debian directory). There are a few other projects consuming upstream tarballs from Debian's archive. I've seen source-based distributions (portage, pkgsrc) using tarballs from there. It would be friendly to not break this for them if we can avoid doing so without too much cost. Upstream tarballs are probably also the easiest way for upstream to provide a signed version of their software which we have tried to encourage (for example by including such signatures in Debian's archive). Ansgar
Re: Git Packaging Round 2: When to Salsa
On Thu, 2019-09-12 at 08:14 -0400, Sam Hartman wrote: > Ian> 1. The maintainer's git repository branch format must be > Ian> documented. Otherwise another contributor has to guess. This > Ian> could be done either by doing maintainer uploads with dgit > Ian> (since recent versions of dgit include the maintainer branch > Ian> format information in the git tags), or perhaps by writing > Ian> something in README.source. > > Makes sense. > My take is discussion on debian-devel strongly supports making it easier > to figure out what branch format people are using. I don't see much value in this requirement (besides additional work). One should look at the repository anyway whan planning to do changes (to match the existing style used); one would naturally see how files are organized. We already had tons of packages shipping a README.source stating "this packages uses quilt, ..." before which I also didn't find very valuable; this seems pretty similar. If dgit provides a program to figure this out, people interested in obtaining the information automatically can just extract and use that. (Using dgit to upload packages is sadly incompatible with best practices around packaging.) Ansgar
Re: Mozilla Firefox DoH to CloudFlare by default (for US users)?
Adam Borowski writes: > On Tue, Sep 10, 2019 at 07:46:57PM +0200, Marco d'Itri wrote: >> Well, no. They cannot without significantly more expensive hardware to >> do DPI and a *totally different* legislative framework. >> (Source: I have been dealing with government-mandated censorship in >> Italy for ~15 years, both at technical and policy levels.) > > I don't understand how blocking by IP would be any more expensive than > blocking by DNS. It's _cheaper_: you read a field in the IP header instead > of doing it in a higher level DNS server. >From the top of my head I can think of several reasons: - For IP-level blocking you need to implement blocking in more places instead of a central place (DNS); also more data needs to be processed in total. Block lists are generally not public and access to them might need different restrictions (for legal reasons). - IP-level blocking leads to more overblocking (anything sharing the same IP); this causes legal problems. So Marco's arguments seem reasonable. >> > * Cloudflare can falsify DNS¹ >> You can use DNSSEC over DoH. > > If implemented. It's probably easier to use DNSSEC with DoH as you avoid broken resolvers at ISP or customer routers that don't speak DNSSEC or not even proper DNS. I've encountered customer routers that knew only about `A` RRs and lied about `PTR` which breaks stuff in interesting ways... Ansgar
Re: Git Packaging Round 2: When to Salsa
Anthony DeRobertis writes: > On 9/12/19 8:57 AM, Ansgar wrote: >> I don't see much value in this requirement (besides additional work). >> One should look at the repository anyway whan planning to do changes >> (to match the existing style used); one would naturally see how files >> are organized. We already had tons of packages shipping a >> README.source stating "this packages uses quilt, ..." before which I >> also didn't find very valuable; this seems pretty similar. > > Working with packages downstream it's nice to have that > documented. E.g., needing to patch something for a weird site > requirement, or backport a fix that isn't a big deal for anyone else > (so likely wouldn't qualify for a stable update), etc. Not everyone > who wants to modify a package is familiar with the multitude of ways > of maintaining packages. As long as you only need to touch the more trivial parts of the package and not the upstream source. There are many more ways to organize upstream sources; more conventions (for different programming languages); more workflows; code style conventions; ... Most of the variance in Debian packaging is much less than what one would encounter outside of the packaging-specific bits. Ansgar
Re: Git Packaging Round 2: When to Salsa
Sam Hartman writes: > If you are a Debian Developer packaging a package for inclusion in > Debian, you should store your packaging information in one repository > per package on salsa.debian.org in the debian group. That is you should > create a repository under https://salsa.debian.org/debian . This allows everyone to do arbitrary changes and presumably upload the package. That is quite a large change that I'm not sure everyone likes. +--- | Direct commits to repositories in the Debian group by any Debian | developer are implicitly welcome. No pre-commit coordination | (e.g. merge-request or mail) is expected. +---[ https://wiki.debian.org/Salsa/Doc#Collaborative_Maintenance:_.22Debian.22_group ] Ansgar
Re: Git Packaging Round 2: When to Salsa mirror
Geert Stappers writes: > On Sun, Sep 08, 2019 at 10:05:17PM -0700, Russ Allbery wrote: >> Higher chance that the repository won't go away. > > Where is Alioth? As retoric question. The repositories on Alioth weren't lost and can still be found archived on https://alioth-archive.debian.org/ > What about Vcs-Mirror-* for actual telling where a mirror is. > In case of `git` is "mirror" a "clone" Arguably in Git everything but the maintainer's local repository might be a mirror. This was even called a security feature as hash collisions might need to be sneaked in there to get included in release tarballs[1]. Ansgar [1] https://public-inbox.org/git/pine.lnx.4.58.0504291221250.18...@ppc970.osdl.org/
Re: Proposed build profile: noinsttests
Simon McVittie writes:
> The name of the profile
> ---
>
> The "noinsttests" name is inspired by previous discussion (on a bug
> asking for more in some package, if I remember correctly?),
> and by ginsttest-runner in the gnome-desktop-testing package[1], which
> is a test-runner for GNOME's installed-tests initiative.
>
> If people strongly prefer build profiles to be singular, then "noinsttest"
> would also be fine.
>
> The conventional names for the build-time options that enable/disable
> GNOME installed-tests are --{en,dis}able-installed-tests (Autotools) and
> -Dinstalled_tests={true,false} (Meson), but I think "no-installed-tests"
> or "no-as-installed-tests", or a version of one of those names with
> words run together, would be inconveniently long for a build-profile.
I think a name without abbreviations like "no-installed-tests" is
better. While it is clear what the name means for people working with
build profiles all the time, a more expressive name might be easier on
people only dealing with them occasionally.
Ansgar
Re: tag2upload (git-debpush) service architecture - draft
Bernd Zeimetz writes: > On 7/27/19 8:16 PM, Rebecca N. Palmer wrote:> As a way to avoid relying > on SHA-1, would it work to have git-debpush >> include a longer hash in the tag message, and tag2upload also verify >> that hash? >> > The other idea would be to convince git upstream to use something > better than sha1 - and after a bit of searching, I found [...] > So I think the best thing to do is to get sha256 working in git and > force the usage of sha256 if you want to sign a tag for upload. That will take quite a while; we would probably need a version of git supporting that in stable. There are also other issues, for example: - Such a service would bypass various sanity checks on the archive side, including various permission checks. - Such a service would need to properly validate the PGP signature. The archive really shouldn't rely on a third-party service for this. (In particular the service in question here doesn't do that as far as I can tell.) Ansgar
Re: Please stop hating on sysvinit (was Re: do packages depend on lexical order or {daily,weekly,monthly} cron jobs?)
Alf Gaida writes: > We need sysvinit for some non-linux things No: Hurd existed for a long time without using sysvinit/sysv-rc. I think sysvinit was only ported to Hurd in 2013 or 2014 (I didn't search much, but found a Summer of Code application from 2013 for this). Having sysvinit might make things a bit easier for Hurd/kFreeBSD, but it's not an absolute requirement for such a port to exist. Ansgar
Re: Debian and our frenemies of containers and userland repos
Marc Haber writes: > Compared to a full VM, a container _is_ smaller. I am not sure whether > the difference is as huge in times where we have kernel same-page > merging though. > > Do we have a build technology that uses containers instead of chroots > yet? I haven't used it so far, but at least "whalebuilder" exists. The gitlab-ci used on salsa.d.o also uses Docker containers; people also build packages using this (mostly for testing though, see for example [1]). Ansgar [1] https://salsa.debian.org/salsa-ci-team/pipeline
Re: Git Packaging Round 2: When to Salsa
Matt Zagrabelny writes: > On Wed, Sep 25, 2019 at 4:09 PM Bernd Zeimetz wrote: >> Why do all things need a recommendation? List the possible places to >> host a repository with their advantages/disadvantages. If somebody is >> not able to decide where to put a package based on that, they might not >> be able to maintain it anyway. >> > > Having sensible recommendations reduces the barrier to entry. Even if you > provide a list with all the pros/cons there is still a decision to be made. > Every decision that needs to be made is a barrier. Reading and > understanding, research and applying context, contemplating and making > decisions. Then the best recommendation is probably "find a team to maintain the package in" and, if that really doesn't work for some reason, "try to find a team maintaining similar packages (language, domain) and consider following whatever they do" to make it easier to interact with these people when asking for advice or so. Ansgar
Re: Integration with systemd
Simon Richter writes: > On Sat, Nov 02, 2019 at 07:40:52PM +0100, Thomas Goirand wrote: >> There's no need to do that. If a backported package is using such >> features, then it just should depend on the correct version of systemd. >> You may have seen that systemd 242 is already in buster-backports... > > Yes, that's one of the questions I have asked: is systemd a core system > component that we want to provide a stable release for, or is it one of the > peripheral packages that users can upgrade to a backported version if they > need a new feature, or has Debian relaxed its standards to accomodate > systemd? I guess it can work the same as for src:linux? That is a core system component, but users can upgrade to a backported version. I don't think Debian has relaxed its standards to accomodate linux. systemd isn't any different from other packages. > Regardless of whether Debian wants to have diverse init systems or > standardize on one, we need to have a clear position on what kind of > support users can expect for the stable release. If a significant fraction > of users runs a backported systemd, can we provide security updates for > them, or will we shunt these users onto a "rolling release" track, and if > yes, who manages that track? Currently the systemd maintainers also maintain a backport for systemd. Ansgar
Bug#943914: RM: firefox-esr [armel] -- RoQA; build-dependency nodejs not available on armel
Package: ftp.debian.org Hideki Yamane writes: > firefox-esr package doesn't migrate to testing but I cannot > find the reason at https://qa.debian.org/excuses.php?package=firefox-esr I believe we should remove firefox-esr/armel to allow the current version to migrate to testing. Ansgar
Re: Integration with systemd
Simon Richter writes: >> > No, and that's not our job. There are a lot of people out there building >> > non-systemd systems. [...] > My point with that sentence is a different one though: a lot of Free > Software exists outside the Linux sphere that does neither anticipate nor > require tight integration with systemd, and that software is still useful, > so it makes sense to ship it. It is niche, but it is the niche we have been > traditionally good at, so I don't see a reason for leaving that and > focusing on an oversaturated field without even marginally preparing. I'm not sure what that has to do with Debian supporting multiple init systems? Nobody is suggesting to not package software that doesn't need tight integration with systemd; most software probably won't. Ansgar
Re: Integration with systemd
Martin Steigerwald writes: > Of course that raises the question on what relationship with a > downstream like Devuan to aim for. Debian has so many downstream distributions, one more fringe distribution doesn't make any difference in relationships with downstream distributions. Besides that, from earlier communication from the 2000 active developers of the Devuan distribution, they were planning to stop importing updates from Debian anyway. Ansgar
Re: libraries depending on interpreters
Niko Tyni writes:
> It's not obvious to me what the ideal setup here should be, when there
> are no other reasons for such a dependency than 'need an implementation
> of the language the library is written in'.
I think there can be another reason: "needs a minimum version of the
interpreter language".
(An "if X, then at least version X-Y of X" dependency that also comes up
in other contexts, e.g. systemd units and the minimum version to support
a given set of directives in them. Debian sometimes tries to express
that with `Breaks: X (<< Y~)`.)
> On the one hand, the library is probably not useful without an
> interpreter for the language, but on the other hand people explicitly
> installing the library (as opposed to an application pulling it in)
> presumably know what they are going to use it with.
But one doesn't always need to use the `/usr/bin/${lang}` interpreter,
but could also use an embedded interpreter in another program (usually
loaded as a shared library). For example, the pl{perl,python,...}
extensions for Postgres.
Though these corner cases might not be worth taking too much
consideration: installing the interpreter only takes up a bit more disk
space.
> - the Python Policy [1] says modules must depend on the default Python
> runtime package: Depends: python3 (>= 3.Y). I suppose this is needed
> at least for the standard library. The dependency scheme seems to be
> pretty consistent in the archive.
As far as I know Python also byte-compiles modules in postinst, so such
a dependency might be required for that alone. Though that tests if the
`py3compile` program (or similar) is actually installed.
Ansgar
Re: Automated removal of RC buggy packages
On Tue, 2019-11-12 at 00:51 +0100, Matthias Klose wrote: > why do you even consider such uploads suitable for unstable? That's > something > which should go to experimental. With packages from experimental, users do not get updates by default even for packages installed from experimental and/or only available via experimental. But for packages like firefox, users should really get updates by default. Unstable is arguably also easier to use than experimental (no extra source entries, no pinning, ...). I believe binNMUs are also mostly only scheduled for packages in unstable. Ansgar
Re: Git Branch Names / DEP-14
Simon McVittie writes: > On Tue, 05 Nov 2019 at 20:40:43 -0800, Russ Allbery wrote: >> My normal use of experimental does not involve maintaining unstable and >> experimental branches simultaneously. > ... >> I know some people do more of a two-branch setup > > One common reason to need to use experimental more actively is if your > upstream has a relatively long-running "latest feature development" > branch that is explicitly not suitable to be in a stable release, such > as dbus 1.odd.z, GNOME 3.odd.z, and Linux/Xorg/Mesa release candidates. In such cases it might be even more appropriate to have branches like "debian/1.odd" if they better represent the actual history, in particular if the "debian/1.odd+2" is not a descendant of "debian/1.odd", but an independent branch. (One can otherwise force the new branch to be a descendant by "fake" merges, but that is not a good idea for various reasons: it creates an incorrect history and confuses tools that now think commits were applies when they really were not.) Ansgar
Re: Secureboot: how to use MOK
Steve Langasek writes: > On Sun, Oct 27, 2019 at 10:45:49AM +0100, Florian Weimer wrote: >> * Thomas Goirand: >> I don't think secure boot provides any benefit at all if you store the >> kernel module signing key on the same machine. > > Generate the MOK certificate with EKU 1.3.6.1.4.1.2312.16.1.2. This > indicates that the key should only be trusted for kernel modules, not for > kernels or other EFI applications (bootloaders etc). The value is honored > by shim, grub (via shim), and the kernel (but not by the firmware - but the > firmware itself doesn't trust the MOK anyway, so this doesn't matter). > > This does not eliminate all attacks that involve getting access to the > private key on the machine; but it does prevent the presence of MOK + DKMS > being used to attack the firmware. I thought the Linux kernel did not call `ExitBootServices()` and this is the reason we have to require all modules to be signed by default. (Or even if it did, this applies to all modules loaded before.) So the Linux kernel should be able to chainload anything, just like shim. So why does this prevent attacks on the firmware? It shouldn't matter if you can sign the kernel or any module run in the same context as the kernel. Ansgar
Re: should all bug reports be filed against /source/ packages?
Guillem Jover writes: > On Wed, 2019-10-23 at 08:32:11 +0200, Ansgar wrote: >> the thread about naming (source) packages reminded me of an other thing: >> Debian's bug tracking system currently (mostly) tracks bugs against >> binary packages and (less often) against source packages. > >> It gets confused if a source & binary package with the same name, but >> otherwise unrelated exist; or when the same binary is built from >> different sources on different architectures; > > I think this is a bogus practice that dak or ftp-masters should > outright reject on NEW processing. The second case might have been > valid in the past, but I don't think it is anymore since we have > versioned provides. I consider these to be bogus because I don't > think we can properly tell the infra which would be which w/o some > kind of manual intervention. It's bogus practice to confuse source & binary package namespaces which too many places in our infrastructure do (including the BTS). >> or when binary and source versions don't match (version tracking really >> should use source versions). > > debbugs at least has the apparent support for source and binary > versions at least at filing time, where you can do: > > Package: binary > Version: version-binary > > or > > Source: source > Source-Version: version-source > > if that does not really do what it's supposed to do, or other parts > get confused, I'd say this is a bug in debbugs that should ideally > be fixed. Trying to implement version tracking for binary versions is so complex and provides so little benefit (IMHO) that it's not worth trying to do so... Please keep in mind that there is no historic version information for binary versions (unlike d/changelog for source history) and that source pcakages can build binary packages with arbitrary versions, i.e. you cannot assume anything about relationship between source and binary versions. Debian's infrastructure also only uses source versions to close bugs... >> In addition there are issues when binary packages get >> renamed (e.g. when libfoo1 gets dropped in favor of libfoo2). > > This is indeed a valid point, and it would be nice to get support to > handle this in an easier way. Say a debbugs command to mass reassign, > or some way to designate the new package as a successor of the old > one, so that the infra could do the reassignment itself, or similar. Why more complicated solutions? >> I believe bugs should always be assigned to source packages as source >> packages are really the unit we use to keep track of packages. > > In some contexts that might be true, but for bug tracking and triaging > just using the source package implies a massive loss of relevant data > and grouping. :/ You can still use other grouping mechanisms (usertags). Binary package names and versions should probably also still be included in the bug report, just as we do for other useful information such as dependencies. >> So I'm wondering if we should start just filing all bug reports against >> source packages? Reportbug could probably be easily changed to use >> `Source: ...` instead of `Package: ...`; more places could follow later. > > I actually find it slightly annoying when getting bug reports relevant > only to a binary package filed against the source package, as that's > something else I need to fix. > > Personally I only ever file against source packages, when the bug is > relevant to the actual source package, say f.ex. the packaging bits or > the upstream build system, something in the actual source, say some > licensing issues, etc. Which bugs don't fall into this category? Not being relevant to all of the packaging bits, build system, actual source, and licensing issues seems to leave very few things open that would not be filed against source packages according to your list. Ansgar
Re: should all bug reports be filed against /source/ packages?
Sune Vuorela writes: > On 2019-10-23, Ansgar wrote: >> So I'm wondering if we should start just filing all bug reports against >> source packages? Reportbug could probably be easily changed to use >> `Source: ...` instead of `Package: ...`; more places could follow later. > > Have you ever maintained source packages where a source produced many > different binaries? I know that even maintainers are confused by Debian's BTS and sometimes don't notice bugs not filed against the "main" package as you would have to visit other pages to notice them. And bugs reports get "lost" when packages get reorganized (renamed, merged, split in different ways). I've seen that happen quite often. > I'm not sure the libreoffice maintainer, nor bug reporters, would > appreciate this. I as a bug reporter would certainly appreciate not having to look into all packages anyway as the bug might already be reported against a different binary. > Back when all the kdepim packages (mail, nntp, rss, calendar, contacts, > ...) were one source, I'm sure dealing with them would have been a > nightmare. We have usertags and other mechanisms that allow grouping bugs in maintainer-defined ways. This is also used by pseudo-packages where we don't have "binaries" to group bug reports by. For packages with too many bug reports, Debian's bug tracker doesn't work already (requests time out when too many bugs exist). Ansgar
Re: [RFC] Proposal for new source format
Simon McVittie writes:
> On Tue, 22 Oct 2019 at 05:22:57 +0200, Bastian Blank wrote:
>> - Files need to be compressed and are recorded as such, which is a hard
>> problem and give rise to tools like pristine-tar and such.
>
> My understanding is that this is deliberate: it means the only layer
> with the hard requirement to be able to cope with malicious/crafted files
> without introducing security vulnerabilities (whether that means arbitrary
> code execution via parser bugs, or denial of service via "zip bombs")
> is the PGP signature verification on the (uncompressed) .dsc. Everything
> else is authenticated before being decompressed, either directly via
> the PGP signature or via the authenticated hashes in the .dsc.
I think there are two separate uses:
- if you want to validate that the upload is as intended by the
maintainer, then a signature of the uncompressed source is
sufficient. (A signature over the compressed source works too if you
do not want to switch to new compression format later.)
- for all other purposes (regular downloads, ...), one would like a
signature over the data that is used, i.e. usually for downloads of
the compressed variant.
kernel.org uses a similar scheme: there are signatures for the
uncompressed tarballs by the maintainer (linux-*.tar.sign). In addition
there is a sha256sums.asc which has strong hashes of the compresssed
files (linux-*.tar.{gz,xz}) and is signed by their archive management
system.
As far as I understand git-archive is fairly good as reproducing
identical uncompressed tarballs at a later time from the git repository.
Ansgar
should all bug reports be filed against /source/ packages?
Hi, the thread about naming (source) packages reminded me of an other thing: Debian's bug tracking system currently (mostly) tracks bugs against binary packages and (less often) against source packages. It gets confused if a source & binary package with the same name, but otherwise unrelated exist; or when the same binary is built from different sources on different architectures; or when binary and source versions don't match (version tracking really should use source versions). In addition there are issues when binary packages get renamed (e.g. when libfoo1 gets dropped in favor of libfoo2). I believe bugs should always be assigned to source packages as source packages are really the unit we use to keep track of packages. So I'm wondering if we should start just filing all bug reports against source packages? Reportbug could probably be easily changed to use `Source: ...` instead of `Package: ...`; more places could follow later. Ansgar
Re: Integration with systemd
Simon McVittie writes: > On Thu, 31 Oct 2019 at 17:51:28 -0700, Russ Allbery wrote: >> I think we should adopt sysusers.d fragments as the preferred mechanism >> for creating system users > > I have been tempted to write a small reimplementation of systemd-sysusers > suitable for init-less containers and sysvinit systems, so that we can > rely on its declarative syntax even on non-systemd systems - even though > I use systemd myself and am happy with it as my init system, so it's > entirely possible that I would never *use* the reimplementation. I think splitting out services that are useful in environments without an init system installed, such as sysusers, might be useful. Possibly also tmpfiles, but without an init system nothing would start the service and it would have to be invoked manually. Maintainer scripts might use it though to setup directories in /var/lib or similar locations. I think we should agree that using them is okay before considering to split them into a separate package from systemd. > I've vaguely considered the same thing for tmpfiles.d, although a full > reimplementation of tmpfiles.d is somewhat more difficult because it's > more featureful. There is already an alternative implementation for tmpfiles.d: https://github.com/OpenRC/opentmpfiles I don't know more than that it exists though. Ansgar
Re: Facilitating external repositories
Paul Wise writes: > On Mon, Nov 4, 2019 at 4:52 AM Guillem Jover wrote: > >> The official archive-keyring packages that use these, I think it's mostly >> for backwards compatibility reasons. > > I wonder if it is feasible to and how the debian-archive-keyring could > migrate from /etc/apt/trusted.gpg.d/ to /usr/share/keyrings/ + > signed-by. Right now it ships keyrings in both places. I would recommend against doing this as long as sources.list is a configuration file: it would need regular updates to change to the new signing key. That doesn't work out of the box. Ansgar
Re: Integration with systemd
Simon Richter writes: > On Wed, Oct 30, 2019 at 05:17:57PM -0700, Russ Allbery wrote: > >> One can individually say that one doesn't care about those features, but >> we just cannot say Debian as a whole should not care about those features. >> It doesn't work. We have to take an affirmative stance on what Debian is >> going to do with software that does care about and use those features; >> whether we're committing to porting it, whether we're going to kick it out >> of the archive (that seems highly unlikely to be viable), whether we're >> going to say that software with a hard dependency on systemd features is >> allowed to only work on systems running systemd, or some other >> alternative. > > From an integration point of view we need to make sure that software that > is installed also works, and I have little trouble when packages declare an > explicit dependency on systemd running as pid 1. Such a dependency doesn't work well with services installed in containers which can still access stuff outside the container. Or services started by a user (who doesn't have to use the provided system integration). > They need to declare a > dependency nonetheless, because init systems have never been Essential, and > also because the dependency likely needs a minimum version. No: init systems have been essential until systemd (or init) replaced sysvinit. Note that there are other cases where we don't have explicit dependencies such as depending on the Linux kernel itself, even though some programs require minimum versions (including glibc). > However, a lot of our software comes from the BSD world and will never > fully switch to systemd, and that software is often used in server contexts > by people who know exactly what they are doing. I don't see why we > shouldn't support these people anymore, especially as they are the ones who > cannot be served equally well by other distributions. Sure, you can just not install any init system. This doesn't require Debian to support other init systems and provide integration with them. > Infrastructure I'd like to see is > > - dh_systemd generating a versioned dependency for the unit files > given That makes supporting other init systems even less possible as everything shipping any systemd service will start to pull in systemd, even though one might not use it to start the service. I think this is not a good idea. > - apt being able to blacklist packages and hide packages that depend on >those Implicitly hiding some packages seems very confusing. Ansgar
Re: Integration with systemd
Russ Allbery writes: > Josh Triplett writes: > >> Part of the problem is that the people interested in sysvinit don't tend >> to care about those features and often argue that others shouldn't care >> either, and the people interested in those features don't tend to care >> about sysvinit. It's difficult to motivate people to work on >> alternatives for a system they already have and prefer. > > *This* is the thing that I feel we need to tackle head-on. Because there > are a lot of reasons to care about those features, but possibly more > convincingly, as Ted points out, *upstreams* care about those features and > are not going to stop caring about those features just because Debian > packagers do not. But many people *don't* care. See the history of consolekit and systemd-shim which nobody wanted to maintain and then were unhappy when they were eventually gone... The argument there has always been "I don't care about these features", but upstream projects used them and eventually systemd was the only implementation. >> It seems evident based on the history of such efforts that there is >> *not* sufficient people/interest/motivation to reimplement the majority >> of systemd features, let alone doing so in a way that meets the >> additional constraints imposed on such solutions. > > I don't think the question has yet been forced. Up through buster, one > has been able to mostly run Debian on sysvinit without doing this work, > with some pain and some gaps and some issues. I think we're coming up to > a point where this issue will be forced because both packaging practices > and upstreams are diverging away from sysvinit support. > > I think there are two questions here: > > 1. Do we as a project want to do the work to leave open the possibility >that such resources will materialize? This means doing things like >defining a subset of systemd unit features that packages can rely on, >and requiring (some? all?) packages not use features outside of that >set. "No" is a possible answer here, but this is a political question >with significant consequences, and I think it should be decided >democratically. Is there someone willing to do the work you envision? If not, what would be the use of deciding about it democratically? I think we shouldn't spend large amounts of resources to provide support for such a hypothetical possibility. I can imaging doing the same for other system interfaces, such as mandating only a subset of the Linux kernel API to be used to support alternative implementations of that API. These already exist, e.g. Microsoft's Linux subsystem (the old one) and I believe BSDs also can pretend to be a Linux kernel. We could also standardize the libc ABI to allow alternative implementations, or at least not use non-standard extensions that make replacing glibc harder. I don't see why systemd should be treated different from other parts of Debian. > 2. Are we as a project *capable* of producing a non-systemd alternative >that's viable? If the answer to question 1 is "no," then this question >doesn't arise. But it's entirely possible that the answer to question >1 is "yes" but the answer to this question is still "no." I don't really believe writing init systems is Debian's goal, just like implementing our own X server or other stuff. That other implementations that use systemd's unit files might be useful, but so far no other implementation has come into existence for years. Ansgar
Re: MBF: make fdisk non-essential
Thomas Goirand writes: > Do you already have a list of affected package? A list of affected packages was attached to the mail. Ansgar
Re: Heads up: persistent journal has been enabled in systemd
On Thu, 2020-02-06 at 12:30 +0100, Martin Steigerwald wrote: > Especially as I found that I did not use journalctl in my daily > practice anyway. Given you wrote earlier that you moved all but one of your machines away from Debian, whatever Debian installs by default doesn't affect you anyway. I don't write on mailing lists of distributions I don't use (or only use a bit) arguing about what is included in their default installation, though one could argue that emacs should be installed everywhere by default. > So why would removing rsyslog from the default install the only > viable approach to solve duplicate logging? If the rsyslog maintainer is also fine with this (which I'll assume he is), I think there's no real problem if that would be done. I might install rsyslog by hand on a few systems then, but that's not a real problem given I already have to install other packages anyway... I have no problem installing a different MTA than Debian's default (exim), my preferred shell, my preferred editor and so on either. Ansgar
Re: Heads up: persistent journal has been enabled in systemd
Marco d'Itri writes: > On Feb 06, Simon Richter wrote: > >> I'd expect servers and embedded systems to be vastly underrepresented in >> both of these statistics, but that doesn't mean these use cases are in any >> way uninteresting to the project. > If you want to argue that the adoption of popcon is not uniform among > different types of systems then you need to prove it. > Expectations and anedoctes are not statistics. That's likely given some libreoffice packages (unlikely to be installed on servers) are at ~40% popcon and I would expect significantly more server installations than desktop ones. So popcon might overestimate sysvinit usage and it might in reality be lower over the total installation base. Ansgar
Re: Y2038 - best way forward in Debian?
Russ Allbery writes: > Ansgar writes: > >> So maybe just recommend people to move to 64-bit architectures and put >> 32-bit applications in a time namespace so they believe they are still >> in 2001 ;-) 32-bit architectures will probably still be useful in >> embedded contexts for a long time and there it might be easier to just >> change the ABI, but for a general-purpose distribution we start seeing >> more and more problems and I don't really see us supporting them as a >> full architecture in 10+ years. > > If we go down this path, can we make cross-grading a supported feature for > the next stable release? I'm sure I'm not the only one who is stuck with > continuously-upgraded i386 hosts who has been wanting to switch but has > been waiting until cross-grading is a little bit less scary. I think the window of opportunity to support this is already over: most people have already migrated to amd64, there is no real incentive for these to spend effort on adding support for cross-grading (besides academic interest, but I for example find adding support for one-time migrations of legacy systems not that rewarding). So you would need to recruit people still running i386 hosts, but for each individual it might be easier to just wait for still existing i386 systems to be decommissioned or just reinstall. (I would likely recommend reinstalling ancient systems for various reasons anyway.) For just avoiding the Y2038 problem, i386 might sort itself out already without additional intervention. From recent popcon reports I get: | Release| i386 | amd64 | i386/amd64 (%) | |+---++| | 1.41 (etch)| 323 | 66 | 4.89 | | 1.46 (lenny) | 958 |440 | 2.18 | | 1.49 (squeeze) | 1753 | 2553 | 0.69 | | 1.56 (wheezy) | 3573 | 9455 | 0.38 | | 1.61 (jessie) | 4687 | 26977 | 0.17 | | 1.64 (stretch) | 4932 | 62619 | 0.08 | | 1.67 (buster) | 3978 | 66169 | 0.06 | | 1.69 (t/u) | 360 | 10445 | 0.03 | |+---++| | Overall| 20728 | 180079 | 0.12 | #+TBLFM: $4=$2/$3;%.2f I'll assume the composition of installations within a release didn't change over time. This shows exponential decay for i386 installations (which is expected) with a factor of ~0.5 for each release. If we consider 10 years in the future or 5 releases, one would expect i386 usage to have gone down to about 0.03 of the current value, that is about 0.002 of installations. Ansgar
Re: Heads up: persistent journal has been enabled in systemd
On Wed, 2020-02-05 at 07:44 +, Scott Kitterman wrote: > Do syslog facilities really have to be addressed by number rather than name? > That seems like a horrible interface. Currently yes. There is an upstream bug asking for a way to specify them by name[1], but nobody implemented it yet. I think a `--facility` option should be fairly easy to implement. Just adapt some code from the existing `--identifier` and `--priority` options, there is already a method to translate facility names to numbers (see calls to `log_facility_unshifted_from_string`). Ansgar [1]: https://github.com/systemd/systemd/issues/9716
Re: Y2038 - best way forward in Debian?
On Wed, 2020-02-05 at 08:33 -0500, Sam Hartman wrote: > Steve, you're presuming that we would not create a new soname for > libc6 on architectures where we want a new time ABI. Isn't the libc ABI for some reason part of Debian's architecture name? uclibc-linux-amd64, musl-linux-i386, i386 are distict architectures after all. So an incompatible newglibc-linux-i386 would be different from i386 as well? Ansgar
Re: Heads up: persistent journal has been enabled in systemd
On Wed, 2020-02-05 at 13:32 +, Scott Kitterman wrote: > > Currently yes. There is an upstream bug asking for a way to > > specify > > them by name[1], but nobody implemented it yet. > > > > I think a `--facility` option should be fairly easy to > > implement. Just > > adapt some code from the existing `--identifier` and `--priority` > > options, there is already a method to translate facility names to > > numbers (see calls to `log_facility_unshifted_from_string`). > > > > Ansgar > > > > [1]: https://github.com/systemd/systemd/issues/9716 > > Thanks. That seems like a pretty basic feature. It is easy to implement, but systemd upstream seems not to like the rather inflexible syslog facilities which don't cover all services people run. Journalctl can display logs for abitrary groups of services, e.g. $ journalctl --since=-1h -u dovecot -u "postfix@*" -u dkimpy-milter -u dkimpy-milter-verify which would give similar results as looking at mail.log. Ansgar
Re: Y2038 - best way forward in Debian?
On Tue, 2020-02-04 at 13:14 +, Steve McIntyre wrote: > What do people think here? Which do you think is the better path? Feel > free to point out things you think I may have missed. We should get > started on this soon - the longer we leave it, the more likely it is > that we'll see 2038 bugs biting people. At least for i386, I expect it to be used mostly for legacy applications (and legacy installations). So breaking ABI by switching to a "new" architecture or by just changing major libraries like libc6 probably diminishes its value so much that there would no longer be any use for it: one could just switch to amd64 instead of i386t. So maybe just recommend people to move to 64-bit architectures and put 32-bit applications in a time namespace so they believe they are still in 2001 ;-) 32-bit architectures will probably still be useful in embedded contexts for a long time and there it might be easier to just change the ABI, but for a general-purpose distribution we start seeing more and more problems and I don't really see us supporting them as a full architecture in 10+ years. Ansgar
Re: Y2038 - best way forward in Debian?
On Wed, 2020-02-05 at 09:55 -0500, Sam Hartman wrote: > > > > > > "Ansgar" == Ansgar writes: > > Ansgar> On Wed, 2020-02-05 at 08:33 -0500, Sam Hartman wrote: > >> Steve, you're presuming that we would not create a new soname > for > >> libc6 on architectures where we want a new time ABI. > > Ansgar> Isn't the libc ABI for some reason part of Debian's > Ansgar> architecture name? uclibc-linux-amd64, musl-linux-i386, > Ansgar> i386 are distict architectures after all. So an > Ansgar> incompatible newglibc-linux-i386 would be different from > Ansgar> i386 as well? > > Not if they are coinstallable. > As an example libc5 and libc6 were both on the i386 architecture. Why wouldn't musl-linux-i386 be coinstallable with glibc-linux-i386? Debian's musl package (on amd64) is coinstallable with libc6; we just don't build anything[1] against alternative libraries. One practical problem is that you don't want to end up linking two different C libraries (or really: any core libraries). I don't really see much difference between libc6 vs libc7 or libc6 vs libmusl (besides applications that require glibc and might not work with musl, but that is a separate issue). This might also mean that musl-linux-i386 shouldn't be a different architecture than i386. If they are different, then maybe libstdc++6- libc6-linux-i386 and libc++-libc6-linux-i386 would need to be different architectures too? Ansgar [1]: Yes, very few exceptions might exist.
Re: How to undo a merged-user installation?
On Wed, 2020-02-19 at 12:58 +0100, Johannes Schauer wrote: > Quoting Guillem Jover (2020-02-18 19:43:53) > > * If you are using raw debootstrap then just pass that option. Or > >just switch to use mmdebstrap which does not have a broken default, > >and is way way faster anyway. > > since you mention mmdebstrap, let me add another reason against getting > merged-usr the way that debootstrap does it. > > The main reason I wrote mmdebstrap was *not* that it's so much faster or > produces smaller tarballs than debootstrap. My main motivation was, to have a > tool that can be used to experiment with a *package-centric* way of setting up > a Debian rootfs from scratch. That limitation in debootstrap exists because Debian currently support both merged-/usr or non-merged-/usr. If Debian would *require* merged- /usr and packages would stop shipping files in `/bin`, then a package could just ship the `bin -> usr/bin` symlinks. Ansgar signature.asc Description: This is a digitally signed message part
Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib
On Wed, 2020-02-19 at 09:39 +, Simon McVittie wrote: > On Wed, 19 Feb 2020 at 09:31:51 +, Simon McVittie wrote: > > I agree that what Guillem is proposing also does not have the property, > > which I think is one that is important to you?, that the contents of the > > root directory are decoupled from /usr (can be set up by an initramfs > > or a container-runner, perhaps in a tmpfs, without detailed knowledge > > of the OS installation in /usr). > > Or perhaps Guillem is intending that in n years' time, when no package > in Debian (not even libc6!) ships files in /bin /sbin /lib* in its > data.tar.*, *then* the maintainer-script-maintained symlink farms in /bin > /sbin /lib* can be replaced by symlinks bin -> usr/bin, etc., without > this resulting in anything dpkg-managed being overwritten or aliased? > > If that's the case, then we get that desirable property *eventually* under > this proposal, but not any time soon. As far as I know the path `/lib64/ld-linux-x86-64.so.2` is part of the ABI (and similar paths on other architectures). So that will have to exist unless we break the ABI. Therefore I assume this is unlikely to happen. I don't think updating *everything* to use `/usr/bin/sh` instead of `/bin/sh` and so on a practically implementable proposal, especially as any local or third party use would need to be updated as well. If you start creating symlink farms in bin/* to usr/bin/*, there is also the risk to get stuck in that state. AFAIR this might be the case with OpenSuSE (not sure, I think that was their state some time ago at least). Ansgar
Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib
Wouter Verhelst writes: > On Wed, Feb 19, 2020 at 06:26:32AM +0100, Marco d'Itri wrote: >> On Feb 19, Guillem Jover wrote: >> > For any pathname that has been hardcoded a symlink can be used for >> > backwards compat, nothing unlike /bin or /sbin here. This looks just >> > like a normal bug from a botched transition, nothing special. >> Creating symlinks in /bin and /sbin DOES NOT result in a merged-/usr >> system, because the content of /usr would not be decoupled anymore from >> the content of /. > > No, but it allows us to *transition* from the current situation to a > merged-/usr situation, *without* breaking the package manager's > expectations. openSuSE tried to transition to merged-/usr that way seven years ago[1]. I booted a recent live system and noted that they are still stuck in the transitional state. I would not recommend following that example. [1]: https://en.opensuse.org/openSUSE:Usr_merge >> A merged-/usr system must have /bin /sbin /lib* symlinks to /usr. > > We can have a policy that by release X, any package installing anything > in /bin /sbin or /lib that is not a compat symlink is RC buggy. > > Once we've reached that point, we can drop the directories and convert > them to symlinks. At that point, dpkg can also ignore any request to > create the compat symlinks, and later on we can make it an RC bug to > create compat symlinks if we wish to no longer support non-merged-/usr. > > Yes, this approach takes more time, but it is an equally valid way to > move from unmerged /usr to merged-/usr, and is the approach that people > are advocating. It's not "the" approach that people are advocating; that would apply that there was consensus that the (singular) approach is the right way. I would recommend *not* following it given it wasn't successful for other distributions. The largest blocker for doing anything is that Debian hasn't agreed on making merged-/usr mandatory anyway. Ansgar
Re: Y2038 - best way forward in Debian?
Arnd Bergmann writes: > On Mon, Feb 10, 2020 at 11:16 PM Florian Weimer wrote: >> There's going to be a _TIME_BITS selector, similar to >> _FILE_OFFSET_BITS. >> >> There was a proposal to have only one API before, but I think the >> agreement was that it wouldn't save much complexity. > > It should be easy to force the _TIME_BITS selection by patching the > glibc headers in Debian though if we want. > > The problem with setting _TIME_BITS=64 only using dpkg-buildflags > but leaving the libc default at 32 bits is that anything that users build > themselves or that ignores the buildflags ends up with a broken ABI > when linking against one of the many libraries that expose time_t > in their ABI. It breaks the other way too: If you have an old library with a time_t in some public ABI, but an application using it sets _TIME_BITS=64, the headers suddenly define a different ABI from the one implemented by the compiled library. If you rebuild the library with _TIME_BITS=64, it changes ABI too and breaks reverse dependencies (ignoring special handling like libc does with versioned symbols). So you cannot just change it by default. I don't understand how this can work unless time_t is *never* exposed by any library other than libc. Ansgar
Re: Heads up: persistent journal has been enabled in systemd
Hi, On Sat, 2020-02-01 at 13:36 +, Steve McIntyre wrote: > Michael Biebl wrote: > > with today's upload of systemd 244.1-2 I finally enabled persistent > > journal by default [1]. It has been a long requested feature. > > > > The package will create a directory /var/log/journal on upgrades and new > > installs, which enables persistent journal in so called auto mode. > > Fine for new installations, but please *don't* do this for > upgrades. Those people with existing logging setups will be surprised > by this. I generally prefer upgraded systems to behave more like newly installed ones, but I wonder if here syslog daemons like rsyslog should ship a journald.conf dropin in lib/systemd/journald.conf.d/00-rsyslog.conf setting `Storage=volatile`. If you have installed a syslog daemon, you likely don't want a persistent journal as well. Upgraded systems that have rsyslog installed won't get a persistent journal this way. The downside is that magic like this might not be easily discoverable and confuse people who for some reason want a persistent journal and syslog. Ansgar
Re: Y2038 - best way forward in Debian?
Steve McIntyre writes: > wzss...@gmail.com wrote: >>Is there the option C? >> >>Draft: >> 1. define time64_t >> 2. for data_struct which act as a part of ABI, define a new data_struct64 >> 3. for function which act as part of ABI, define a new version func64. >> 4. patch all packages to use time64_t instead of time_t step by step. >> 5. set a time as deadline (2030?), and then treat all packages >>haven't finished >> the migration as rc-buggy. >> >> Since we have enough time, we can patch all packages in that period. > > It's possible, but... > > The problem here is that we have many thousands of packages to work > on, glibc up through other libs to all the applications that use > them. It's invasive work, and likely to take a very long time. Since > it's work that would *not* be needed for 64-bit architectures, we're > also likely to see push-back from some upstreams. > > 2030 is also too late to fix the problem - people are already starting > to see issues in some applications. We want to make a difference soon: > the earlier that we have fixes available, the fewer broken systems > will have to be fixed / removed / replaced later. For arm* and mips*, we mostly seem to be talking about special-purpose systems where just switching to a new architecture/port doesn't seem to be that much as a problem as for i386. I think rebuilding the world and breaking ABI might thus be acceptable there. i386 seems different. I think option C above would be the only realistic proposal so far to fix the time_t problem for (parts of) i386, but if glibc upstream doesn't want to expose two interfaces then i386 will probably just break. Ansgar
Re: Is running dpkg-buildpackage manually from the command line forbidden?
On Thu, 2020-01-16 at 08:50 -0800, Daniel Schepler wrote: > These are where I found that having extra packages installed during > the dpkg-buildpackage run either failed or resulted in broken > packages. (Some examples of the type of thing I mean: #948522, > #887902.) If you build outside a controlled environment the set of additional packages are just a small part of your worries. There are others such as - more creative solutions to satisfy Build-Depends, - anything in /usr/local, - anything in /opt (if PATH or other configuration includes /opt), - which alternative is active, - diversions, - local configuration, including environment variables (PATH, variables changing behavior of the Python interpreter, ...), - modified files, - local packages that do not exist in Debian's archive, or - modified packages. In many cases this will not be a problem, but trying to make any guarantees that the package building process will still work reliably seems to be a lost cause to me. It is fine to ask a maintainer to include some `--disable-something` flag if this is easy to do and reasonably likely to be a problem, but I would not expect people to actively spend effort on this. Ansgar
Re: Is running dpkg-buildpackage manually from the command line forbidden?
Johannes Schauer writes: > My advice would also be to switch from debootstrap to mmdebstrap because the > latter is able to create a chroot with only Essential:yes packages in it while > debootstrap includes Priority:required packages as well. Alternatively, > debootstrap could gain a variant only installing Essential:yes packages and > their dependencies (why doesn't it have that already?). Debian doesn't support systems that don't have "required" packages installed. So buildds should have them installed. Policy states: "Removing a required package may cause your system to become totally broken and you may not even be able to use dpkg to put things back, so only do so if you know what you are doing." "Essential" packages just have additional requirements (in particular essential packages must work even in the "unpacked" state). Ansgar
Re: opentmpfiles & opensysusers, and its use in the Debian policy
Thomas Goirand writes: > My proposal is for Debian to standardize on: > /bin/tmpfiles > and: > /usr/bin/sysusers Why rename things? > I'm not sure why > there's both /bin/systemd-sysusers and /usr/bin/systemd-sysusers, and > which one should be used. For the same reason there is /bin/bash and /usr/bin/bash probably? Ansgar
Re: opentmpfiles & opensysusers, and its use in the Debian policy
The Wanderer writes: > On 2020-01-02 at 08:14, Thomas Goirand wrote: >> As I wrote, no need to complain, but act. >> >> https://salsa.debian.org/debian/opentmpfiles > > For me (with no salsa account, therefore not logged in; I don't know if > that makes any difference), this page states "The repository for this > project is empty". The repository has only tags, but no branches. This seems to confuse GitLab's web interface, but one can still clone the repository and checkout tags. (Probably Thomas forgot to push the branches to GitLab.) Alternatively the upstream repository without the Debian packaging bits can be found at [1] (might be a mirror, not sure). Ansgar [1]: https://github.com/OpenRC/opentmpfiles
Re: opentmpfiles & opensysusers, and its use in the Debian policy
The Wanderer writes: > However, as it remains the case that they are shipped in the same > package as /bin/systemd, and as I gather (mostly from this thread, I > think) that some of the ways they are expected to be invoked probably > rely on having systemd-the-daemon running They don't rely on the `/bin/systemd` binary at all, and I'm fairly certain that this was mentioned in the thread (and previous) and not the opposite. There was something about dh_installsystemd currently only running systemd-tmpfiles in maintainer scripts if systemd run, but that can be changed should Debian choose to use tmpfiles for more generic purposes. > this does not entirely > obviate my concerns related to needing to have systemd-the-package's > daemons present in order to gain access to these facilities. I'm happy to have helped overcome these concerns. Ansgar
Re: RFC: Replacing vim-tiny with nano in essential packages
Gunnar Wolf writes: >> > Well, yes. But while mostly everybody who reads this will be >> > moderately proficient with the basic subset of vi, I don't know >> > anybody who'd know how to drive ed (I have done it, but I surely don't >> > remember how to). >> >> It's not about the size of the editor package but more about using an >> editor which causes less build issues. > > ...and which still works for the users. Yes, ed _can_ be used. But I > really do not think including ed would satisfy a regular user in need > to unbreak a minimal system... A regular user shouldn't have to deal with a minimal system, but should have a less minimal editor installed or use a rescue system. Ansgar
Re: trimming changelogs
Hi, Adam Borowski writes: > Of files named changelog.*, top offenders are: [ Trimmed list to duplicates ] > 223009 ncurses-bin:changelog.gz > 223009 ncurses-base:changelog.gz > 223009 libtinfo6:changelog.gz > 210621 libc6:changelog.Debian.gz > 210621 libc-bin:changelog.Debian.gz > 145889 libapt-pkg6.0:changelog.gz > 145889 apt:changelog.gz > 145271 passwd:changelog.gz > 145271 login:changelog.gz > 93638 libudev1:changelog.Debian.gz > 93638 libsystemd0:changelog.Debian.gz > 63670 logsave:changelog.Debian.gz > 63670 libss2:changelog.Debian.gz > 63670 libext2fs2:changelog.Debian.gz > 63670 libcom-err2:changelog.Debian.gz > 63670 e2fsprogs:changelog.Debian.gz We should probably also not ship the same changelog in multiple packages, especially when one depends on the other. > Seems like a tempting area to trim... I agree it would be fine to trim changelogs a bit; no real opinion on how much to keep (maybe at most three years or so?). We certainly don't need 20 years of changelog in any binary package by default in my opinion. > On the other hand, changelogs are valuable. Unlike some folks on IRC > I wouldn't want to tightly trim all packages. Unlike minbase or > prio:important, your average 5GB install doesn't care about a few megs > here and there. Thus: do we want to trim manually or globally? Special cases for packages are annoying, so I would just apply the same policy for all packages (by default). Ansgar
Re: documenting on how not starting a daemon upon installation
Marvin Renich writes:
> * Marco d'Itri [200310 11:23]:
>> The modern and simple solution is "systemctl mask", as long as you do
>> not need to care about the 0.6% of systems which do not use systemd.
>> If you are doing this for your own systems then you obviously know if
>> you can rely on systemd or not.
>
> I don't believe this is correct, though I could be wrong. I believe
> policy-rc.d is sufficient for both systemd and sysvinit systems, and
> that it is necessary for _packages_ that only ship an init script and
> not a service file, regardless of the init system in use.
>
> Can you tell me,
>
> A) Does systemctl mask work for packages that do not have a systemd
> service file when systemd is the init system?
It should work: init.d scripts just get a .service file generated that
calls `ExecStart=/etc/init.d/${something} start` (and some somewhat
appropriate settings). You can find the generated units in
/run/systemd/generator*; see also man:systemd-sysv-generator(8).
You can also change other settings for init-based services by creating a
`/etc/systemd/system/${something}.service.d/local.conf` just like for
native systemd services.
> B) Can systemctl mask be run on a subdirectory that you are about to
> chroot into, but have not yet done so?
I would expect something like `systemctl --root=/chroot mask
${something}.service` to work. If it doesn't work, that's probably a
bug.
Ansgar
Re: RFC: Replacing vim-tiny with nano in essential packages
John Paul Adrian Glaubitz writes: > And I assume, once we have fixed vim everywhere, it will be broken again > at some point due to the fact vim upstream is continuously adding features > which is why it's no longer suitable being an editor to be shipped in a > minimal installation. And Debian ships vim-tiny, not vim, as part of the minimal installation. That the same source package also builds other versions doesn't really matter for vim-tiny. The only problem you mentioned was vim-tiny (arch: any) depending on vim-common (arch: all) and these sometimes getting out of sync on Debian Ports. I don't think that is a good reason to switch editors and there are other ways to work around that problem. But if we really wanted a minimal editor: `ed` is still there with an Installed-Size: 116 kB and no external dependencies besides libc6. It also works without fancy terminal features. Or have debootstrap not install any editor. But if I remember correctly that idea wasn't popular. Ansgar
Re: Bug#969942: ITP: tty-share -- Terminal sharing over the Internet
[ Resent to include d-devel@ in Cc: as this is a reply to there, but the wnpp pseudo-package doesn't have d-devel@ as maintainer... ] Holger Levsen wrote: > On Wed, Sep 09, 2020 at 06:27:28AM +, Francisco Vilmar Cardoso > Ruviaro wrote: > > * Package name: tty-share > > Version : 0.6.2 > > Upstream Author : Vasile Popescu > > * URL : https://github.com/elisescu/tty-share > > * License : Expat > > Programming Lang: Go > > Description : Terminal sharing over the Internet > > > > tty-share is a very simple command line tool that gives remote > > access to a UNIX > > terminal session. > [...] > > tty-share connects over a TLS connection to the server (runs at > > tty-share.com), > [...] > > this pretty much sounds like this should go into contrib, rather than > Debian main. Or is there free server code as well? There is no requirement for the "other side", be it firmware, network services, ... of communication software (clients, drivers, ...) to be released under a DFSG-free license to be in Debian (main). It seems counter-productive to have such a requirement as it only would mean more people have to use contrib/non-free. Ansgar
Re: Tagging in Salsa -> upload: status?
On Wed, 2020-08-19 at 10:15 -0700, Sean Whitton wrote: > On Fri 14 Aug 2020 at 09:04AM +02, Ansgar wrote: > > There are also other issues such as the system seeming to accepting > > uploads from known-compromised keys last I looked at it, though > > maybe security experts disagree how much of an issue this is in > > practice. > > If what you say about known-compromised keys is true (I think I might > know what you're alluding to but am not sure), then obviously such a > bug could easily be fixed. Well, I can't fix it without creating dgit-ng (and setting up infrastructure for that) as dgit upstream won't accept patches from me. Ansgar
Re: Proposal: Allowing access to dmesg for users in group adm
On Mon, 2020-08-17 at 15:50 +1200, Matthew Ruffell wrote: > I propose that we restrict access to dmesg to users in group 'adm' like so: > > 1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel. > 2) Following changes to /bin/dmesg permissions in package 'util-linux' > - Ownership changes to root:adm > - Permissions changed to 0750 (-rwxr-x---) > - Add cap_syslog capability to binary. > 3) Add a commented out '# kernel.dmesg_restrict = 0' to >/etc/sysctl.d/10-kernel-hardening.conf That grants additional rights to the `adm` group that it did not have before, for example to clear the dmesg buffer: $ dmesg --clear works after adding `cap_syslog` to the dmesg binary whereas it did not work before. Ansgar
Re: How much data load is acceptable in debian/ dir and upstream
Tobias Frost writes: > my 2 cents: debian/ should not be used for much data: It will be duplicated > by the upload > of every package revision. So (being extreme now), having several hundreds of > MiB would > be quite expensive in terms of storage overheade. > > I have not idea how much "much data" is, for sure to be defined by FTP > masters. > > For another 2 cents, if data would exceed the size of the debian.tar.gz by a > magnitude or two, I would go for a source package. > > Summing up to 6 cents, my feeling is that storing data in debian/ is not the > best > approach. Have you talked to upstream to include the data in their source. > After all, > this would benefit the whole community if upstream has a good test suite? The 3.0 (quilt) source format also supports using several upstream source tarballs. Firefox uses this for shipping source for the program (firefox_X.orig.tar) and translations (firefox_X-l10n-*.tar). Ansgar (not having looked into this issue before)
Re: DEP-14: renaming master to main?
On Mon, 2020-06-22 at 22:13 +0100, Colin Watson wrote: > For a while I'd thought that it was relatively harmless in comparison > to > full-blown uses of master/slave metaphors, but I saw some analysis of > the history recently that pointed out that git got it from BitKeeper > which did in fact use a master/slave metaphor [1]. > [1] https://mail.gnome.org/archives/desktop-devel-list/2019-May/msg00066.html You might be interested in [2] as well. Speculation is often wrong. [2]: https://mail.gnome.org/archives/desktop-devel-list/2020-June/msg00023.html Ansgar
Re: Tagging in Salsa -> upload: status?
Sean Whitton writes: > Ian and I implemented something along these lines last summer and it's > available to try from the archive; here is how: > <https://spwhitton.name/blog/entry/tag2upload/> > > As to the current status: FTP Team members objected to having > uploader-signed git tags on dgit.debian.org be the canonical record of > an uploader's intended source package (rather than uploader-signed .dsc > files stored on other servers), and they objected to the ways in which > the system relies on git SHA1 hashes. > > I still believe that the design is sound and deploying the system can > and should go ahead, but we could not overcome the disagreement. There are also other issues such as the system seeming to accepting uploads from known-compromised keys last I looked at it, though maybe security experts disagree how much of an issue this is in practice. Ansgar
Bug#978636: move to merged-usr-only?
Package: tech-ctte X-Debbugs-Cc: debian-devel@lists.debian.org Hi, as suggested in [1], I would like to see Debian to move to support only the merged-usr filesystem layout. This would simplfy things for the future and also address the problem with installing files under aliased trees that dpkg has to do for both variants to be supported. merged-usr has been the default for new installations since the last release and (as far as I am aware) no world-breaking bugs have happened since; some environments such as buildd chroots still do not use merged-usr. I would like to ask the technical committee to decide whether we want to move to merged-usr-only. It seems to be a case of overlapping jurisdiction (6.1.2 in the constitution). I'm not asking the committee to decide on a concrete technical implementation for this. Obviously we would need to also implement a migration path for legacy installations for a move to merged-usr-only to be implemented. This also isn't relevant for Debian 11 (bullseye), but I would like to have enough time in the Debian 12 (bookworm) cycle. Ansgar [1]: https://lists.debian.org/debian-devel/2020/11/#00232 continued in December: https://lists.debian.org/debian-devel/2020/12/#00386
Re: move to merged-usr-only?
On Sat, 2020-11-21 at 02:01 +, Paul Wise wrote: > On Fri, Nov 20, 2020 at 7:35 PM Simon McVittie wrote: > > > Package-by-package migration touches a large number of packages > > By my count there are 1712 binary packages from X source packages > installing things outside /etc /usr /var The goal is to have /bin and /usr/bin to have identical contents. If one wishes to achieve that via symlinks for every single binaries, you not only need symlinks in /bin for binaries previously there, but moved to /usr/bin, but also for binaries that already are in /usr/bin. So one would need a new /bin/python3 -> /usr/bin/python3 symlinks in addition to the /bin/bash -> /usr/bin/bash symlink discussed here. This affects many more packages. Starting a 10-year[1] migration for the small part (move bash to /usr/bin, add /bin/bash -> /usr/bin/bash) symlink, then maybe[2] start *another* *multi-year) migration after that, and then getting there isn't a great outlook for me. (At that time migration to merged-/usr for the remaining systems would no longer be a worry either way as presumably only very few installations without merged-/usr would exist anyway and no migration would be needed, i.e., like the i386 -> amd64 cross-upgrade nobody really worries about any more.) Ansgar [1]: https://lists.debian.org/debian-devel/2018/11/msg00354.html [2]: cf. OpenSuSE or suggestions to never do that and instead wait until nobody uses /bin/sh any longer. If you suggest to do package-by-package migrations, please at least argue why Debian wouldn't end in the same in-between state as SuSE.
Re: move to merged-usr-only?
On Sat, 2020-11-21 at 11:07 +, Paul Wise wrote: > On Sat, Nov 21, 2020 at 10:33 AM Ansgar wrote: > > > The goal is to have /bin and /usr/bin to have identical contents. > > So one would need a new /bin/python3 -> /usr/bin/python3 symlinks > > Those seem unnecessary to me, since we have $PATH and nothing should > be using /bin/python3 at this stage. We have $PATH, yet there are bugs that come from using `/usr/bin/rm` as mentioned in my inital mail. There is no reason to assume that the same doesn't happen in the other direction. I've already seen people using `#!/bin/python` as interpreter a long time ago (before Debian had merged-/usr by default). One can try to educate people that this is wrong because some binaries were historically only available in /usr (due to too small root disk, but a larger disk for /home), but why bother when we can just get rid of the problem and even already have done so partly? Or you can say we ignore that error class for a few more years and only address half of it and then the other half in X years. Ansgar
Re: apt ignoring check-valid-until flag
On Thu, 2020-12-17 at 00:47 +0100, John Paul Adrian Glaubitz wrote: > On 12/17/20 12:36 AM, Paul Wise wrote: > > * snapshot could gain a re-signing service (#763419) > > That would be absolutely awesome. Whom do I throw my money at? It doesn't seem too complicated to implement and could be developed independent from snapshot.d.o: If any Release.gpg/InRelease file is requested: - Retrieve the original Release+Release.gpg/InRelease files. - If there is a valid signature from any previous archive key: - Generate new signature (Release.gpg/InRelease) and store it in some cache. (Bonus points if this keeps the original signature if possible.) - Return the generated Release.gpg/InRelease. - Otherwise: - Return some HTTP error? Or the unmodified Release.gpg/InRelease? Any other files: - Redirect to normal snapshot.d.o Only some storage for recently-requested Release.gpg/InRelease files would be needed. The service could run independent from snapshot.d.o and redirect most requests there. Maybe the same could be done for archive.d.o? I might be interested to experiment with this as it seems reasonably small project to implement. :-) Ansgar
Re: apt ignoring check-valid-until flag
On Fri, 2020-12-18 at 01:15 +, Paul Wise wrote: > On Thu, Dec 17, 2020 at 10:03 AM Ansgar wrote: > > (Bonus points if this keeps the original signature if > > possible.) > > Two separate signatures is possible for Release+Release.gpg, just > rename the latter to .old, but what can you do for InRelease? Is it > possible to have multiple signatures in one blob of signing data? Is > it possible to take an existing signature and add a second one to it? > Can the same thing be done for Release.gpg? Do apt, gpg and gpgv cope > with this sort of thing? We do that for stable's InRelease and Release.gpg: it is signed by keys held by ftp-master and stable release team and we merge the signatures. For detached signatures you can even just concatenate two ascii-armored signatures and GnuPG will find all signatures (we did that for older releases). InRelease requires merging them. For GnuPG to check multiple signatures they must all use the same message digest (i.e. all signatures must use SHA-256, not one SHA-256 and another SHA-512 or similar), but for a signature-refreshing service we probably don't want new signatures for old Release files to use MD5 or SHA-1 even when the old signature did. For our use case consumers must accept when /any/ signature is valid for this (not /all/!); this doesn't make it less secure as we just require a single signature anyway so an attacker having one could drop additional signatures. Other applications might want to explicitly require more than one valid signature. Ansgar
move to merged-usr-only?
Hi, I would like to propose to plan to move to support merged-usr-only over the following releases. The motivation is bugs like [1] where upstream developers just use `/usr/bin/rm` (or other binaries, or user scripts using /usr/bin/bash, or ...) unconditionally; this was already a motivation to adopt merged-/usr as a default for me. As far as I know nothing broke catastrophically over the last releases with merged-/usr. Alternatively, a team could form that preemptively looks at such issues and fixes them, ideally upstream. So a possible idea would be to: - For Debian 12 (bookworm): make it mandatory to migrate old systems to merged-/usr on upgrade. Possibly by allowing the existing usrmerge program to run from the initramfs. - For Debian 13 (trixie): packages should no longer install to /bin, /sbin, /lib, but to the respective locations under /usr. Ansgar [1]: https://bugs.debian.org/973853
Re: move to merged-usr-only?
On Fri, 2020-11-20 at 10:19 +0100, Adam Borowski wrote:
> On Fri, Nov 20, 2020 at 09:35:42AM +0100, Ansgar wrote:
> > I would like to propose to plan to move to support merged-usr-only over
> > the following releases. The motivation is bugs like [1] where upstream
> > developers just use `/usr/bin/rm` (or other binaries, or user scripts
> > using /usr/bin/bash, or ...) unconditionally; this was already a
> > motivation to adopt merged-/usr as a default for me.
> >
> > As far as I know nothing broke catastrophically over the last releases
> > with merged-/usr.
>
> Unless you look at dpkg or attempts at speeding up bootstrap.
>
> See
> https://salsa.debian.org/glibc-team/glibc/-/commit/49d137c4392cb1144f2313f78f31466aaa169b75
> for an example.
>
> As far as I know, dpkg maintainers consider usrmerge to be unsupported,
> and trying to make my own NIH deb installer I see why.
The good news here is that shipping bash as /usr/bin/bash (instead of
/bin/bash) solves that problems as dpkg will just install it under
/usr/bin/bash. No aliasing over symlinks involved.
You also don't need any special handling for anything in /bin, /sbin as
nothing would be installed there (all packages ship the files in /usr).
> > So a possible idea would be to:
> >
> > - For Debian 12 (bookworm): make it mandatory to migrate old systems to
> >merged-/usr on upgrade. Possibly by allowing the existing usrmerge
> >program to run from the initramfs.
>
> Counterproposal: replace debootstrap with mmdebstrap, which is many times
> faster -- and doesn't support usrmerge at all, or at least disable usrmerge
> in debootstrap in default.
That is unrelated. Also you don't need special usrmerge support once
all packages ship files under /usr.
If you care about speed: not calling sync() way too often would
probably make dpkg significantly faster and possibly reduce the total
time the system is in an incinsistent state (of half-updated packages),
reducing the chance of system crashes breaking stuff ;-)
If you see the aliasing problem as a large issue, we could also try to
ship files in /usr already for bookworm.
> > - For Debian 13 (trixie): packages should no longer install to /bin,
> >/sbin, /lib, but to the respective locations under /usr.
>
> Moving stuff with no mandated path is a good idea, yes. Alas, it's been
> massively complicated by usrmerge being a thing, and thus you can't just
> ship the file in a new location as you risk a path conflict.
You can just ship /usr/bin/bash instead of /bin/bash in an updated
package.
> So let's make it so a canonical path to a file never includes a directory
> symlink; if you insist on /usr/bin/rm then /bin/rm should be
> /bin/{rm->/usr/bin/rm} not /{bin->usr/bin}/rm
Why ship /bin/rm at all? Seems too complicated and just ends in the
half-migrated state that SuSE was in last I checked.
> > [1]: https://bugs.debian.org/973853
>
> That's a piece of software for which upstream is Red Hat. The number of
> people developing on RPM distros is rapidly falling, so this is less and
> less of an issue.
Well, other distributions like Debian, Ubuntu, ... also use merged-/usr
these days.
Ansgar
Re: move to merged-usr-only?
On Fri, 2020-11-20 at 11:12 +0100, Ansgar wrote:
> On Fri, 2020-11-20 at 10:19 +0100, Adam Borowski wrote:
> > So let's make it so a canonical path to a file never includes a directory
> > symlink; if you insist on /usr/bin/rm then /bin/rm should be
> > /bin/{rm->/usr/bin/rm} not /{bin->usr/bin}/rm
>
> Why ship /bin/rm at all? Seems too complicated and just ends in the
> half-migrated state that SuSE was in last I checked.
Oh, and I forgot: I don't think adding new /bin/python3 ->
/usr/bin/python3 symlinks (repeat for everything else in
/usr/{bin,sbin} and possibly some parts of lib) to all packages is a
desirable goal.
One point is *not* having / to know about the contents /usr. That was
discussed many times in the past and I don't think we need to revisit
it.
Ansgar
