Re: Package marked for autoremoval due to closed bug? [and 1 more messages]
Ian Jackson writes ("Re: Package marked for autoremoval due to closed bug? [and
1 more messages]"):
> Steve, could you please do this for *all* the time_t transition RC
> bugs?
IMO things are currently ON FIRE.
If no-one else has put this fire out by 24h from now, I will attempt
to find which are the relevant bugs and downgrade them all.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Package marked for autoremoval due to closed bug? [and 1 more messages]
Steve Langasek writes ("Re: Package marked for autoremoval due to closed bug?"):
> Migration to testing is largely out of control of the maintainers at this
> point, it's very much dependent on folks rebootstrapping armel and armhf
> against the new library names. Should these bugs be downgraded again to
> important severity?
Yes.
It seems we have consensus on this.
Paul Gevers writes ("Re: Package marked for autoremoval due to closed bug?"):
> For bookkeeping purposes, please usertag downgraded bugs with user
> release.debian@packages.debian.org and usertag time_t-downgrade.
Rather than every maintainer affected by unactionable autoremoval
warnings[1][2] doing this by hand:
Steve, could you please do this for *all* the time_t transition RC
bugs?
That will reduce the scope for individual slips and mistakes,
fulfilling Paul's request:
> Please be careful with downgrading RC bugs.
[1] IMO unactionable autoremoval warnings are extremely undesirable.
The autoremoval system has two purposes: one is to get rid of things
that are in the way of other work, or just rotten. Another is to spur
action where action is needed. Action by a responsible maintainer, or
failing that by anyone else.
An unactionable autoremoval warning represents, at best, a robot
shouting at a human to do something that cannot be done. That can
lead to many humans from afar all turning up being confuseed at the
same problem trying to "help" (or at least, to try to stop the
screaming). If the autoremovals were to actually occur, it would be
automated destruction of non-broken stuff.
To preserve autorm's usefulness, unactionable autoremoval warnings
must be very rare. In this situation, Debian's processes have failed
to ensure this, and there hasn't been an effective backstop.
I suggest that when a widely-applicable problem is generating
unactionable autoremovals, the whole autoremoval system should be
suspended.
The problem is particularly severe when an underlying cause is that
some package, which contains the underlying RC bugfix, isn't
migrating. This seems to be becoming more common.
[2] In my case src:dgit depends on git-buildpackage. The autoremoval
robot wants to remove git-buildpackage because of the time_t bugs
against rpm, xdelta, and pristine-tar. One root cause is that
src:dpkg isn't migrating because of #1066952.
The logic of the autoremoval system is that as an affected maintainer
I ought to go and involve myself with #1066952.
And all of this is nonsense since surely we're not going to autorm
git-buildpackage, but right now we have a giant klaxon saying that's
what's going to happen.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
debvm for autopkgtests with multiple host?
Hi Helmut and others. Some time ago we had a conversation on debian-devel about how to make autopkgtests that spawn multiple nodes and communicate between them, eg for testing network protocols[1]. To summarise that discussion: at that time the best available solution that worked in ci.d.n seemed to be to write an ad-hoc script to run the tests in qemu; three packes had done that, each separately, with complex scripts with many moving parts. I saw debvm, and wondered if it was suitable for this purpose. But, then I looked at its debian/test/control and I see that the tests are marked as flaky.[2] So maybe it isn't reliable enough :-/. I have other questions too, particularly to do with the way I would need autopkgtest to be able to influence package selection in the nested testbeds. Everyone else: has there been any other progress on the multi-node autopkgtest problem ? Thanks, Ian. [1] https://lists.debian.org/debian-devel/2023/01/msg00059.html https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=908274 [2] https://salsa.debian.org/helmutg/debvm/-/blob/main/debian/tests/control -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Multi-host networking software, autopkgtests
Ian Jackson writes ("Re: Multi-host networking software, autopkgtests"):
> For now I'm working on an adhoc thing that uses
> - Restrictions: needs-root
> - overlayfs to make an editable clone of the fs provided by autopkgtest
> - chroot
> - ip netns
> - apt-mark and apt-autoremove to get rid of unwanted deps
>
> It's not particularly pretty but it's not much code and I feel I'm
> making progress. I'll report back here when I've succeeded - or
> failed :-).
This worked very well on my laptop but it has been going quite badly
in the CI. The CI environment is really quite different from the
real one (sometimes in what seem like strange ways).
Right now I am having the following impossible thing occurring. [1]
This code:
tunfd= open("/dev/net/tun", O_RDWR);
if (tunfd < 0) sysfatal("open /dev/net/tun");
r= fcntl(tunfd, F_GETFD);
if (r==-1) sysfatal("fcntl(tunfd,F_GETFD)");
is resulting in this output:
2023-01-10T22:58:46.142075+00:00 ci-010-c4ebefae hippotatd[153]:
error: ipif stderr: userv-ipif service: fatal system error:
fcntl(tunfd,F_GETFD): Bad file descriptor
The executable which prints this message is not multithreaded and has
no signal handlers. I suspect ? ? CI ? container ? hates tun ? ?,
or ? ? overlay fs on /dev ? strange effect on /dev/net/tun ? ?.
(Also in the test I have that *doesn't* do the pid namespace and
chroot thing, service(8) seems to have been disabled. Maybe I need to
add a Restriction for that?)
I looked again at the links you helpfully provided. That does look
like it would be doable, but it's decidedly nontrivial.
Each of those scripts has its own ad-hoc answer (or not) to questions
like "cope with the file:// apt urls not working" "map the Debian
architecture to a qemu binary" "configure the within-qemu environment
so we can do anything to it" "manage the lifetime of the qemu" etc.
I'm considering a third option: "disable the tests in debci" :-/.
Ian.
[1]
Test script source code
https://browse.dgit.debian.org/hippotat.git/tree/adt/acommon?h=archive/debian/1.1.5%2bexp7
Recent test log (full of distracting crap):
https://ci.debian.net/data/autopkgtest/unstable/amd64/h/hippotat/30238962/log.gz
Source code for "ipif" program experiencing the EBADF error
https://browse.dgit.debian.org/userv-utils.git/tree/ipif/service.c?h=archive/debian/0.6.1-2#n711
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Multi-host networking software, autopkgtests
Johannes Schauer Marin Rodrigues writes ("Re: Multi-host networking software,
autopkgtests"):
> there already exist a number of autopkgtest scripts that start a qemu virtual
> machine and then run some script inside of it by connecting to it via ssh.
> Maybe this can be helpful to you for now:
Thanks. I considered this but it seemed overkill (and it won't
inherit the dependency versions selected by autopkgtest).
For now I'm working on an adhoc thing that uses
- Restrictions: needs-root
- overlayfs to make an editable clone of the fs provided by autopkgtest
- chroot
- ip netns
- apt-mark and apt-autoremove to get rid of unwanted deps
It's not particularly pretty but it's not much code and I feel I'm
making progress. I'll report back here when I've succeeded - or
failed :-).
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: propose: provide "docker" package as docker, not wmdocker
Hideki Yamane writes ("propose: provide "docker" package as docker, not
wmdocker"):
> I'd like to propose wmdocker package would rename its source
> package from docker to wmdocker, and then docker.io package
> provides docker binary package and transitional docker.io package.
Do we have a wiki page which gives a procedure for renaming a source
package ?
Source package renames are rather disruptive because we key a lot of
stuff off the source package name. At least bugs need to be
reorganised, and possibly other things. To reuse a source package
name and then upload the new package with dgit, it's necessary to ask
a dgit repo admin to do special by hand adjustments. etc.
This all seems like it could do with a checklist that we can at least
add things to each time we discover a brokenness we should have
avoided...
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Multi-host networking software, autopkgtests
Paul Gevers writes ("Re: Multi-host networking software, autopkgtests"):
> I guess this is best discussed in https://bugs.debian.org/908274 (added
> in the To)? Maybe with Wouter and other interested parties?
Hmmm. Well, a way of doing this "officially" in autopkgtest would be
nice. But:
Think such an "official" thing ought to allow the specification of
different dependencies for the different hosts in the test. And I
don't much like the mini-DSL suggestion. Maybe it would be better to
offer the test script an adt virt server interface to control the
"other" hosts.
This all seems very complex. I definitely want to have something
working before something like that could exist. Also, I think it
would be a good idea to do something ad-hoc, ideally in a number of
packages, to gain experience so we know what shape the "official"
thing ought to be.
I think therefore that I need to pursue some kind of within-testbed
nesting, as an interim approach at the very least. I was hoping that
someone else had solved (part of) this problem already...
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Multi-host networking software, autopkgtests
I have two packages which do vpn-like things (hippotat, secnet) which
I want to add autopkgtests for. The two packages have similar kinds
of requirements for their tests.
Ideally, I would:
* Somehow have two test nodes ("hosts")
* With their own /etc and /var (or, relevant parts thereof,
but it would be better to have two completely separate hosts
so I can test that the client package works even if you don't
have the server package installed, etc.)
* With their own networking setups
* With some kind of network connection between them
All of this would have to be set up from the autopkgtest test script,
which would need to be able to run commands in either node.
And ideally it would be easy to run the tests from my normal dev
environment too (without having to, say, install docker).
Ideally it would let me properly test the service startup (init
scripts, etc.) too for a full integration test, but if necessary that
can be done in a separate one-host test, since the software will
*start up* just fine even if it doesn't have a peer.
I guess I could do something ad-hoc with mount and network namespaces
and overlay filesystems. But it feels like this problem must have
been solved already ?
The part I'm not sure how to do ad-hoc is dependency management. An
autopkgtest ought to use the packages desired by the autopkgtest test
runner. So far the best option I can think of is to declare in the
autopkgtest control file *all* the relevant packages needed for any of
the two test nodes and the setup scripts; in each node, unshare the
namespaces enough that I can run apt; manually uninstall the
not-needed dependencies, and run apt autoremove.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
Thanks to everyone for the comments and review. I have written things up on the wiki: https://wiki.debian.org/BuildProfile/upstream-cargo and added the entry here: https://wiki.debian.org/BuildProfileSpec Please comment here, if you think any of this doesn't make sense. Thanks, Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
Helmut Grohne writes ("Re: Proposed `cargo-upstream` dpkg-buildpackage etc.
build profile"):
> On Thu, Dec 15, 2022 at 11:44:34PM +, Ian Jackson wrote:
> > I'm not sure precisely how this feature could (or should) be made
> > available to *all* application packages in a central way. Having
> > tools like debcargo automatically add the profile to the build deps
> > produces a lot of bloat. I'm hoping there is a better way.
>
> I think this becomes the crux of the matter. I see basically two
> distinct ways to move forward:
...
> What are your thoughts on updating Build-Depends?
I think that in packages where the maintainer has deliberately chosen
to support this build profile, the maintainer should annotate the
Build-Depends.
> B
>
> In this case, the profile description should be more clear about the
> intended scope (rust application packages).
Yes.
Is "application package" precisely the right term? It could also
apply to: packages which provide shared objects (eg a .so plugin);
ABI/API libraries (eg trad C libraries whose innards are now rewritten
in Rust); packages which provide an executable that isn't an
application.
The possible scope includes any package whose .deb contains
executables (including possibly WASM, as well as native code) made by
compiling Rust. It might even include packages where Rust is used
only as part of the build system.
One way to describe the exception would be to say something like
(except Rust library packages)
or
(except Rust library packages themselves)
> I also do not see us automating the update to Build-Depends.
I agree but for different reasons to yours. I don't want this done to
every Rust leaf package routinely, which would be the natural end
point of automating this.
> > This could be made easier. Maybe tools like sbuild could have a
> > sepaarate option to disregard build deps matching a wildcard pattern,
> > or something.
>
> The idea is good in principle. Let me do a brief excursion into
> Multi-Arch for comparison. By default, we have an architecture
> constraint on dependencies. It can be lifted either by M-A:foreign on
> the target of the edge or by :any on the source of the edge. A build
> profile is a bit like :any and annotates the source of an edge. Your
> proposed wildcard pattern would be a target annotation for build
> profiles in this analogy while the current profile implementation has no
> way to do such target annotations.
Yes. The information would ideally be centralised: "if you have a
Build-Dependency matching librust-*-dev, disregard it with this
build-profile". So the edges would be "annotated" but implicitly.
Failing that I guesss we could add a general facility of this shape
Disregard-build-depends-with-build-profile: upstream-cargo
a la M-A.
> This idea is also applicable beyond rust. With the nopython build
> profile, we might want to automatically annotate Python dependencies.
> Unfortunately, we are now in a place where a significant portion of
> packages uses Python modules during build (e.g. sphinx) while not
> producing any Python modules or extensions as output. As such, we cannot
> mechanically tell how such a dependency is being used. Maybe a package
> later contains a rust crate that is not shipped, but only used during
> build. In this case, we wouldn't want to annotate the relevant
> dependencies either.
The analogy with "nopython" is rather misleading. (You could have
"norust" too; not sure if that's in the list right now.)
With nopython, we want to *avoid doing the Python things at all*. But
"the Python things" here isn't "all Python things" - it's "certain
Python things that appear in the outputs". So that can't be done as a
blanket exclusion on B-d.
With upstream-cargo, we want to *do all the Rust things, but get them
all from upstream crates.io*. This should affect *every* dependency
on a cargo package (except ones that are within the source package
itself).
So if a package that previously didn't do any Rust things, gains a
Rust crate that is not "shipped" (to /usr/share/cargo/registry), then
this buid profile does become nontrivially applicable. (Of course
this build profile is trivially applicable to packages which don't
involve cargo at all).
The reason why the upstream-cargo profile ought not to be provided for
a Rust library crate is not becuase it's incoherent or wrong in a
narrow sense.
A Rust library crate typically *does* actually run rustc during rules
build, as a check:
https://deb.debian.org/debian/pool/main/r/rust-cxx/rust-cxx_1.0.73-1.dsc
And in principle it is perfectly coherent to say "please build this
source package, but do the check with upstream Rust Dependencies".
But this is not *useful* to support as a build profile. Narr
Re: Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
Helmut Grohne writes ("Re: Proposed `cargo-upstream` dpkg-buildpackage etc.
build profile"):
> Thank you very much for immediately taking this to the list.
You're welcome.
Thanks for the review and the penetrating questions.
> I think that you imply here that all of the rust libraries would be
> annotated . Do you confirm?
Yes.
> Yes:
>
> I caution that we increase our (uncompressed) sources by 200KB for
> adding this. This assumes changing every package, but the scope
> isn't entirely clear, see below. Do you confirm that this is the
> intention and that you think this increase is fine?
No, this is not my intention. And IMO that increase would be
undesirable.
This facility is certainly cxompletely useless for a Rust library
package - those whose .debs are the Rust text and metadata, for
building other programs against. Those things don't even have "real"
Rust build-dependencies - the Rust build deps are just there for
testing etc. And there is no use to anyone building such a package if
they are using upstream crates.io packages.
So at the very least we're limiting this to leaf packages containing
binaries. And, probably even then, this will only be useful when the
Debian or upstream maintainers want this hybrid or stepping stone
workflow.
I was imagining that this would be done ad-hoc by a maintainer when
they had a reason to do it, not that it would be supported by even
every leaf package. In some cases it might be a transitional state.
I do think it would be useful for the central tooling to offer this
build profile for every package, but I don't think it is appropriate
to annotate all the Rust build depends for every package this way.
Tools like debcargo ought *not* to add this build profile to the Rust
deps.
If this profile turns out to be widely desired for "leaf packages at
random", filtering (or dummy-satisfying) the build dependencies should
be done some other way.
> Which packages should support this profile? I see the value for
> applications. Do you also intend it for libraries? If yes, how do they
> benefit?
They don't.
> How do you intend to transition packages to support this profile?
> Should that happen on an as-needed basis? Should it happen as a mass
> commit? Do you want it mass uploaded?
As needed. No mass commit, no mass upload, no automated change.
> Given all of the mails in the thread thus far, you've convinced me:
> * That the requested feature is useful.
> * That it should be ecosystem-specific.
> * That it needs to be easy to use (and that build profiles satisfy this
>requirement).
>
> On the bike coloring front, I do prefer putting the package ecosystem
> last (i.e. upstream-cargo > cargo-upstream) for the consistency reasons
> that you gave in a reply.
I think I'm sold on that name question.
> I would also question the "upstream" part as it wasn't obvious to me
> initially. Good alternatives that aren't too long are not easy to come
> by, but maybe you have a reason to reject "external"? I think "external"
> would more strongly highlight that a build is no longer self-contained.
I don't have strong feelings about this. I chose "upstream" because
that seems to be the term of art that people working in this space
use, to distinguish the general public package repo, from the Debian
one.
> > However, this could be a generally useful feature for Debian's cargo
> > tooling to support, and I think it could do so in a general way so
> > that this profile would be available in most Debian packages
> > containing Rust code, without package-specific work. Whether to
> > implement such a feature is a matter for the maintainers of dh_cargo
> > et al.; IMO the build profile registration is useful even ad-hoc,
> > without any general feature.
>
> Yeah, having this supported generically seems very useful. It would be
> nice to have a supportive reply from one of the dh-cargo maintainers
> here. (I do not see this as a requirement.)
>
> I note that "without package-specific work" implies that you wouldn't
> attach build profiles to Build-Depends, which was my initial question.
> I'll stop second guessing here and wait for your answer.
I'm not sure precisely how this feature could (or should) be made
available to *all* application packages in a central way. Having
tools like debcargo automatically add the profile to the build deps
produces a lot of bloat. I'm hoping there is a better way.
But even without modifying the build dependencies, it is a lot easier
to cause a build to run without the stated build deps being satisfied,
than it is to do violence to the package build system.
This could be made easier. Maybe tools like sbuild could have a
sepaarate option to disregard build deps matching a wi
Re: Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
Wouter Verhelst writes ("Re: Proposed `cargo-upstream` dpkg-buildpackage etc.
build profile"):
> I have just one question: why make this rust-specific? I think a similar
> thing might be useful for golang packages (who also don't support shared
> libraries), or, heck, even Perl (if I'm willing to test that the package
> works while I have the not-yet-packaged dependencies in my ~/perl5/lib)
You are absolutely right that many other upstream (language-specifc)
package managers will have analogous situations.
I think *each* of these should have their *own* build-profile, rather
than one portmanteau one. If a package has both Rust and NPM
build-dependencies, say, a user may very well want to use Rust
dependencies from Debian and NPM dependencies from upstream, or vice
versa.
The profile name should be named after the upstream package manager
being influenced, not after the programming language, since some
languages have several.
So I think that we will probably want
cargo-upstream
npm-upstream
golang-upstream (is this the right name?)
etc.
Maybe this means that the name ought to be the other way arond, so
they all group together in the table, like the `no*` options.
upstream-cargo
upstream-npm
upstream-golang (is this the right name?)
I don't want to speak for those working with these other package
managers, so I don't propose to add all of those right away. But
establishing the precedent will hopefulloy be helpful for them
So for now I'm proposing to add just the cargo one. If someone
working with Node packages (say) tells me "we want this for npm too"
then great and we should do that right away. Otherwise we should
wait until ti's wanted.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
Jonas Smedegaard writes ("Re: Proposed `cargo-upstream` dpkg-buildpackage etc.
build profile"):
> Similar pain for other upstream ecosystems as well - I know about npm
> for NodeJS modules, but imagine it is similar for Java and others as
> well.
Yes.
> What is the benefit of introducing a standardized flag for this
> relatively narrow scope, compared to doing non-standardized fetching of
> crates _before_ package build and building with those embedded?
> Example of doing that is here: https://salsa.debian.org/debian/helvum
> (essentially doing `cargo vendor --versioned-dirs debian/vendorlibs`).
IDK what precisely you mean there, and you've liked to the repo as a
whole so I'm not sure what to look for. Maybe you mean "what's wrong
with pretending to vendor the dependencies" ?
Whatever approach is taken, it has to be controlled somehow. For
example, the paths to dependencies need to be adjusted, or the use of
the Debian cargo wrapper enabled/disabled.
That control can be done by: (i) modifying the package source code
(which is much more a pain, even if it's a toggle in a single place),
(ii) ad-hoc environment variables or something (which don't survive
sbuild) or (iii) a build profile.
ISTM that this kind of "build this package in s funky way" situation
is precisely what build profiles are good for.
> If there is really a need for a standardized flag, then what is the
> benefit of a narrow one specific to cargo, compared to a more general
> "fetches-source-during-build" that would be suitable also for e.g.
> NodeJS fetching npm modules?
Wouter made the same point - I will reply there, to both the CC lists.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Proposed `cargo-upstream` dpkg-buildpackage etc. build profile
I would like to add the following entry to the table of build profiles in BuildProfileSpec: Profile anme: `cargo-upstream` Can cause changes to built binaries, including functional chnges. (Y) Should not cause changes to set of build debs. (N) Description: Obtain Rust dependencies from crates.io, not from Debian; use a `Cargo.lock` from the source package if there is one. Will usually cause cargo to make network accesses. Ideally, behaviour of the resulting programs will be the same, but this is not guaranteed. This is useful when developing packages which are to be uploaded to Debian, but whose build dependencies are not yet available in the targeted suite. It allows a Debian maintainer or user to more easily rebuild the package in a different suite to the one targeted, and can allow unification of upstream (crates.io-based) and Debian development without needing to carry a build system patch. Particularly, it will sometimes ease the task of getting (leaf) packages in Debian, as other aspects of the packaging can more easily be tested separately from working on getting the dependencies in. This build profile ought only to be used in packages that are intended to appear in Debian (or a derivative) with .debs built from in-distro dependencies (ie the dh_cargo-style local cargo replacement repository, containing). In such packages, obtaining dependencies locally from within-distro is the default behaviour without -Pcargo-upstream. Invocation of this build profile will generally only be appropriate in the context of manual development, or package-specific near-upstream release workflows. Certainly this profile ought not to be activated for uploads to Debian, as it makes the package build depend on out-of-distro inputs (and relies on external network access at build time). For now, I expect to use this only in an ad hoc fashion in affected packages. (I have a number of candidates.) However, this could be a generally useful feature for Debian's cargo tooling to support, and I think it could do so in a general way so that this profile would be available in most Debian packages containing Rust code, without package-specific work. Whether to implement such a feature is a matter for the maintainers of dh_cargo et al.; IMO the build profile registration is useful even ad-hoc, without any general feature. Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Half the world being removed
Paul Gevers writes ("Re: Half the world being removed"):
> On 02-09-2022 13:00, Ian Jackson wrote:
> > I wonder if it would be possible to detect a sudden large increase in
> > the number of autoremovals, and stop the autoremoval system instead of
> > causing blaring klaxons for everyone in the project ?
>
> I disabled the cron job that sends out mail yesterday, so the massive
> klaxons shouldn't go off (or are there klaxons I'm not aware of).
Well, personally I have a grep-excuses cron job that nags me daily
about autoremovals, becaue the official cron job is rather late.
Also it shows up in tracker.d.o for every package, DDPO, etc.
I think to properly silence the alarm it's necessary to prevent the
decision to autoremove, not just the warnings. (There are other
obvious possible difficulties ...)
> And
> indeed such a check would be worth while. But to be fair, the code is in
> Perl and I would be afraid that by adding the check I introduce more
> bugs than I solve.
Do you have someone who will do code review for you ? I hereby
volunteer. (Send me mail off-list, or something.)
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Comments on proposing NEW queue improvement (Re: Current NEW review process saps developer motivation
Sean Whitton writes ("Re: Comments on proposing NEW queue improvement (Re:
Current NEW review process saps developer motivation"):
> On Sat 27 Aug 2022 at 04:22PM +02, Vincent Bernat wrote:
> > It does not seem to work. Either people don't want to do that, either the
> > FTP
> > team is too picky on the candidates.
>
> Some combination of both, but I don't think I'm suffering from bias if I
> say that it's at least 80% the former. Very few people who say they'd
> like to be trained confirm they'd still like to once they've had a look
> at the docs for trainees, and after that, hardly any do enough trainee
> reviews for the other team members to feel confident they can let them
> at it on their own.
I am in this picture. Some years ago now I volunteered. I was
introduced to the internal ftpmaster documentation and processes. At
the time, these documents were not even published - including,
astonishingly, some elements which read like a manifesto. (I don't
know if these documents are published nowadays.)
What I saw was very far away from what I had expected or hoped and
expected to see - especially in terms of process and culture. In
particular it was far away from Debian's usual norms of transparency,
but there were many other problems.
I deecided I couldn't work with it. I could, of course, have tried to
be the change I wanted to see in the world. But, empirically, I'm not
the best person to be trying to lead organisational change in Debian.
As an institution, ftpmaster has been very successful in establishing
and maintaining its own norms and culture. (It does help of course
that it is a tremendously powerful team, whose cooperation is needed
by almost every developer.)
This has upsides: I'm personally strongly aligned with ftpmaster's
primary goals, and very supportive of most of their controversial
decisions (especially, the ones about what counts as source code).
But: the difficulties we have with ftpmaster are very deep-rooted, and
not simply a lack of volunteers.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: Half the world being removed
Paul Gevers writes ("Re: Half the world being removed"):
> On 02-09-2022 07:27, Andrey Rahmatullin wrote:
> > On Thu, Sep 01, 2022 at 11:04:38PM -0500, Steven Robbins wrote:
> >> Suddenly half the packages are marked AUTOREMOVE; many due to gcc-12 and
> >> zlib.
> >> The related two bugs are months-old.
> >>
> >> Why are things suddenly being removed??
> > Both are key packages per
> > https://udd.debian.org/cgi-bin/key_packages.yaml.cgi so it must be some
> > recent problem that causes the things to ignore that.
>
> I'll look into it tonight (UTC+2). I did make one change to udd
> yesterday. I thought it was safe because I just reverted an exception to
> allow scikit-learns removal.
>
> There are probably bugs involved.
Thanks.
I don't want to add to the stressball but: this isn't the first time
that we've had malfunctions which want to remove very large numbers of
packages. Obviously this is a complicated algorithm and I understand
that there will be bugs.
I wonder if it would be possible to detect a sudden large increase in
the number of autoremovals, and stop the autoremoval system instead of
causing blaring klaxons for everyone in the project ?
That might make the failures less disruptive. (And would avoid having
everyone pile-on.)
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
mailman2-python3
Mailman3 is quite a different beast to Mailman2, and is RFH (#998223). I'm currently running a Debian LTS mailman on stable, so that I can avoid the mailman2-to-mailman3 transition, which looks like being no fun at all, whether from my point of view as sysadmin, or that of my listowners. But, there is a python3 port of Mailman2: http://github.com/jaredmauch/mailman2-python3 I think I want to be running this (although, maybe after it has stablised some more). So I think it would be nice to have this packaged. I might well have some time for this, although I would appreciate help and guidance. (Python is not my strong suit.) Thijs, you were the last maintainer, and AFAICT Chris, you were the last person to upload mailman2 other than as LTS support. What is your opinion ? Thijs, do you want to be involved ? I think the next steps would be to try to make some package suitable for experimental. Regards, Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
LTO and ABI compatibility
I have just received Bug#1015348 reporting that adns doesn't work with LTO (link-time optimisation). How does LTO work with ABI compatibility, which we rely on very heavily ? Eg, my reading of the spec is as follows: if I add members to an enum in a new library version, making a combined program containing translation units with the old enum, and ones with the new enum, is UB.[1] But that is precisely what we do when we run new binaries against old libraries. I think src:adns only does things which are justified by traditional ABI compatibility assumptions (albeit, that in some parts of the build it makes these assumptions when linking statically, as well as when linking dynamically). So what, precisely and formally, are the rules ? I feel entitled to demand a fully precise and formal specification of the rules, because it is precisely fully precise and formal and expansive readings of C's literally-incomprehensible[2] specifications which are being used to justify miscompilations (so-called "optimisations"). Frannkly, I think enabling LTO by default is a mistake. The performance benefits are not likely to be worth the bugs silently introduced across our codebase. If there are particular programs that would benefit from it, by all means enable it in those cases. Ian. [1] Assuming that the enum type is used in a relevant way. [2] If anyone doubts that the C specification is literally incomprehensible, observe, for example, the existence of research papers with titles like "Towards a formal semantics for C", or indeed the absolutely hilarious discovery that the specification forgot to define the meaning of assigments when the assignment target was written in parentheses, and that no-one noticed this for decades. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: secnet_0.6.2_multi.changes REJECTED
Firstly, thanks to ftpmaster for your thorough review of this package.
For transparency, I am CCing this reply to debian-devel, and quoting
the whole of the REJECTED mail. There does not seem to be anything in
here that ought to be private. Please let me know if there is
somewhere better. (I considered -legal but it didn't seem
appropriate.)
FTAOD, I am not, with this mail, asking -devel for a peer review of my
package's licensing status, nor of the ftpmaster decision; just using
-devel as a catch-all list.
Sean Whitton writes ("secnet_0.6.2_multi.changes REJECTED"):
> +--+
> | REJECT reasoning |
> +--+
>
> Another team member identified that there is code in this package
> under a number of different licenses other than GPL-3+, but that is
> not specified in sufficient detail in d/copyright. That contravenes
> both Debian Policy and the terms of those licenses.
My apologies. You are completely correct. I don't understand how I
came to think that the approach I took was sufficient. I guess it is
a long time since I prepared a package with so many different bits and
pieces in it.
> They also pointed out that there is some code from StackOverflow,
> which is not GPL-3+.
I think this is not right? I think you are referring to
`argparseactionnoyes.py`. As I documented in the file header, it is
part of StackOverflow's TOS that contributors grant a CC-BY-SA 4.0
licence for the snippets they upload, which would make the
contributions GPL3+-compatible. My file comment gives the relevant
references. [1]
It seems to me that StackOverflow have chosen this approach (making
the upload licence part of the TOS) precisely to enable people to copy
code fragments out of StackOverflow into their own projects. ISTM
that in (unless it appears that the posting StackOverflow user did not
write the code in question), we should be able to rely on that.
Can you please confirm whether the opinion of the ftpmaster team, that
is not sufficient? If it is not sufficent, I guess I will find
someone to write a clean room implementation of this 22-line class.
> I noticed that b91dec.{php,awk} have no license information at all.
As you observe, these files as provided by upstream do not themselves
contain licence information. But the file base91-c/README (which is
provided by upstream) says, amongst other things:
All source code in this package is released under the terms of the BSD license.
See the file LICENSE for copying permission.
And these files were (according to their copyright notice) written by
the same author and clearly part of the base91-c project. I think
this licence therefore applies also to the files b91dec.{php,awk}.
> +--+
> |Other comments|
> +--+
>
> Your changelog mentions changes to comply with modern Policy, so please
> consider upgrading the standards-version too.
Thank you. I appear to have omitted to do this when doing my
standards update review.
> Shouldn't subdirmk be a separate package?
Well. It is designed to be "git-subtree"'d into one's package. That
is the way the upstream package uses it.
It would be possible to make it a separate package and build-depend on
it, at the cost of some additional work. The upside would be a very
small amount of disk space saving, and largely theoretical saving of
work in case of a need to do a security update for subdirmk (which
seems unlikely to be critical since it's build system software which
is designed to execute its input) - and that all only in the case
where a second package in Debian uses subdirmk.
It seemed me best to me to defer this work until subdirmk becomes more
widely used.
Thanks,
Ian.
[1]
https://www.chiark.greenend.org.uk/ucgi/~ianmdlvl/git?p=secnet.git;a=blob;f=argparseactionnoyes.py;h=a7eef1c46345be27eaa90a17858bc52a3f60;hb=HEAD
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: proposed MBF: packages still using source format 1.0
Guillem Jover writes ("Re: proposed MBF: packages still using source format
1.0"):
> Something I might want to see though (although I hold not much hope
> for) is a possible move away from the default behavior when no
> debian/source/format is present, as I think that gives bad defaults
> for newcomers or inexperienced users, and even there just emitting
> warnings tend to be ignored. Possible alternatives could be, either
> erroring out, or changing the default format depending on say a
> dpkg-compat level, or similar, I don't know, have not thought this
> through though. But explicitly marking sources as 1.0 (as has been
> warned for a long time now) would of course keep working as of right
> now.
Thanks for the reassurances. (I have snipped much I didn't feel the
need to comment on but, it was all welcome.)
What you say above makes sense to me. I'm not sure what an
appropriate timescale would be.
Would you welcome implementation of a "3.0 (diff)" format which
contained a tarball plus a single diff, arranged to be capable of
representing every git tree object ? There would have to be some
massaging, I guess, mostly because of symlinks. That would be
pareto-better than 1.0-with-diff.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: proposed MBF: packages still using source format 1.0
Lucas Nussbaum writes ("Re: proposed MBF: packages still using source format
1.0"):
> As explained in https://lists.debian.org/debian-devel/2022/03/msg00165.html
> I proceeded with the MBF for packages that match
> not (debian_x or (vcs and vcs_status != 'ERROR' and direct_changes))
> or, maybe easier to read:
> (not debian_x) and ((not vcs) or vcs_status == 'ERROR' or (not
> direct_changes))
>
> I did not file bugs for packages that are likely to use a VCS-based
> workflow (category (2) in the mail pointed above, or in
> https://udd.debian.org/cgi-bin/format10.cgi)
At least the following packages of which I am the maintainer or
sponsor were includined in the MBF, despite the fact that they are 1.0
native packages with Debian revision:
its-playback-time
spigot
vm
vtwm
chroma
Clearly the it aakes no sense to have filed bugs saying "please switch
to this other source format" when the other source format cannot
represent the package.
Additionally, there were a further 9 packages where I am the
maintainer or uploader where the current version does not have a
Debian revision. They could to be switched to "3.0 (native)" for the
better compression support.
However, given that I perceive that:
- there is a campaign to abolish 1.0
- there are important use cases where 1.0 is needed
- the campaign to abolish 1.0 is being prosecuted anyway
I have deliberately chosen to continue to upload even pure-native
packages as 1.0, to try to slow this process down in the hope that the
situation improves and/or people stop trying to forbid useful things.
I maintain all my packages with dgit. I presume that the vcs status
you are referring to is just that the Vcs-Git header is out of date
since the closure of alioth, or perhaps that there isn't one.
I hope thius clarifies.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: proposed MBF: packages still using source format 1.0
Ian Jackson writes ("Re: proposed MBF: packages still using source format 1.0"):
> But I see now that the MBF has gone ahead anyway.
>
> I spent some time trying to help by setting out the factual
> background, but it seems that Debian is not interested in facts. I
> don't know why I bother.
For example, consider a package maintained by a sponsee of mine:
Debian is not upstream, so it has a Debian revision. The package is
maintained in git, and the source package is very small and it is not
uploaded frequently. So we use a native source format. This means
that we must use format 1.0 because dpkg hates 3.0 native with debian
revision.
My sponsee has responded to this but report by adding a
debian/source/format to their package and asking me to upload. Of
course, I cannot do so. I must instead ask my sponsee t revert the
change.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: proposed MBF: packages still using source format 1.0
Ian Jackson writes ("Re: proposed MBF: packages still using source format 1.0"):
> Sean Whitton writes ("Re: proposed MBF: packages still using source format
> 1.0"):
> > [questions]
...
>
> The situation here is complicated.
>
>
> The tl;dr is that
>
> * there are several situations where 1.0-native is the best answer,
> * there are several situations where 1.0-with-diff is the best answer,
>
> The root cause of both of these situations is that 3.0, sadly,
> is not always better in every respect than 1.0.
After I wrote that, there was a further excvhange where multiple
people replied to me "why are you doing that" - some in very
unpleasant tones of voice.
Answers were given, including by a former DPL (whom you may observe
is not someone I am on speaking terms with).
But I see now that the MBF has gone ahead anyway.
I spent some time trying to help by setting out the factual
background, but it seems that Debian is not interested in facts. I
don't know why I bother.
Ian.
Re: proposed MBF: packages still using source format 1.0
s what Sean is talking about here: > Ian has some cases where something that is representable in git is not > representable using 3.0 (quilt) but is representable using 1.0. I don't > have those cases to hand; Ian, could you summarise, please? Currently, I think diff cannot represent changes to symlinks. git can store symlinks and represent their targets, and changes to their targets. Ian. [1] By "problems" I mean "some software did a wrong thing", not "I am offended by your breach of my notions of propriety". -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: ftpmaster review reply Re: Comments regarding chroma_1.18-1_multi.changes
Sean Whitton writes ("Re: ftpmaster review reply Re: Comments regarding
chroma_1.18-1_multi.changes"):
> It would be disingenuous to claim that as a result of this single perl
> script, the whole chroma /package/ "requires software outside of main to
> function". So I think it is fine to accept to main indeed. However, I
> would like the opinion of a more experienced ftp team member. So I've
> removed the internal note I'd put on the package in NEW, so that someone
> else can more readily take a look.
Thanks. I see this has been ACCEPTed now. Thanks for your work.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
ftpmaster review reply Re: Comments regarding chroma_1.18-1_multi.changes
Hi. Thanks for looking at this package.
Because I believe in Debian's value of transparency I decided to CC
this reply to a public list. I didn't think there was a better list
than -devel. Readers of -devel will find references etc. below [1].
Sean Whitton writes ("Comments regarding chroma_1.18-1_multi.changes"):
> From reading work/resources/README, it seems to me that the
> reverse-engineerings of old games and the conversion scripts need to go into
> contrib, because they're useless without a copy of the original ROMs of those
> games?
I'm quite surprised by this comment.
Firstly, everything in that directory is completely DFSG-free in
itself.
Secondly, almost all of its contents are input files and scripts which
are actually used by the main build system to process the DFSG-free
input files (svgs, etc.) into the files which are shipped in the
.deb. So an integral part of the source code for the DFSG-free .deb.
The only thing which is useless without non-free ROMs is the script
resources/browser/convert2chroma.pl [2]. Obviously therefore this
script is not run at build time and is not shipped in the .deb.
The difficulty you are having here seems to be simply that this one
DFSG-free script, not shipped in any .deb, and not run during the
package build, is not useful as part of a completely-DFSG-free
workflow.
Are you really telling me that we have to strip out from the *source
package*, fully DFSG-free ancillary files which are shipped for
convenience by upstream in the same source tree ? Merely because they
are not used in Debian and don't ahve fully Free uses ?
By that rule any script (or maybe even documentation) in any source
package which is there to help work with proprietary data or on
proprietary systems would have to be thrown out (and the corresponding
source package laundered).
I don't see how this would benefit our users or protect Debian or
anything. And there must surely be many contrary examples of this in
Debian. It is very common for upstreams to provide ancillary stuff in
source packages which we in Debian don't use or ship. [3] They do
this for everyone's convenience and it causes no trouble. Until now :-/.
I hope my explanation is sufficient to get this package accepted.
Thanks,
Ian.
[1] I am replying here to ftpmaster comments on source package
"chroma" which I uploaded as sponsor on the 18th of January. Simon
Tatham, CC'd, is the upstream author and Debian maintainer. The
un-ACCEPTed source package can be obtained by Debian Members with
dgit --for-push clone chroma
as tag archive/debian/1.18-1 (at least unless it is REJECTed).
[2] I have doubled checked this with Simon, the upstream author.
[3] Should we DFSG-launder the Windows support out of our compiler
source packages ? That sounds like fun.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Re: git-buildpackage to be autoremoved due to python2 transition
Andrey Rahmatullin writes ("Re: git-buildpackage to be autoremoved due to
python2 transition"):
> Yes, and as far as I can see after pydoctor migrates the only problem with
> gbp will be python-rpm in debian/tests/control, which is probably
> unneeded.
Oh.
> > the py2 rot seems wider including in gbp itself.
>
> Where exactly?
I looked again and I was looking at an old version. Sorry for the
noise. I still think we have problems with these
937132 nevow: Python2 removal in sid/bullseye
938622 tahoe-lafs: Python2 removal in sid/bullseye
which I think are brought in by pydoctor...
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
git-buildpackage to be autoremoved due to python2 transition
FYI. The widespread impact of this not so apparent because git-buildpackage is typically used ad-hoc by maintainers, rather than via (Build)-Depends. Relevant packages and bugs: 943107 git-buildpackage: Python2 removal in sid/bullseye 937132 nevow: Python2 removal in sid/bullseye 938622 tahoe-lafs: Python2 removal in sid/bullseye Bugs which you may notice which are now not so relevant any more because they have been fixed in sid (but not yet migrated): 950216 [git-buildpackage] missing xsltproc autopkg test dependency Fixed in sid; migration blocked by FTBFS due to pydoctor breakage (#949232). When pydoctor has migrated, reattempting build (eg by re-upload) should fix this. 949232/948831 [pydoctor] needs to depend on cachecontrol 952546 [pydoctor] d/copyright & DFSG issues 937421 [pydoctor] Python2 removal in sid/bullseye My personal interest in this is that dgit Depends on git-buildpackage so I am early in the firing line. However, unfortunately my python skills are very weak and I doubt my blundering efforts [1] to try to mitigate the situation would would be helpful. I can at least do the digging to see what is actually still wrong here. Thanks to Anthony Fok for fixing pydoctor but the py2 rot seems wider including in gbp itself. Ian. -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
bootstrap.min.js in pydoctor
For -devel, context is that Anthony Fok just uploaded a new upstream version of pydoctor (a tool for extracting API docs for python modules) in order to fix a couple of upstream bugs. Anthony, thank you very much for your work to help fix one of our (mutual) indirect dependencies. Unfortunately the new pydoctor has some DFSG issues (#952546, CC'd). I am hoping that -devel can advise what the conventional approach is to the package containing a sourceless copy of bootstrap.min.js. I'm guessing that the answer is to strip the sourceless file from the package, and have the binary package contain a symlink into the file tree of some other package which contains an appropriate bootstrap file ? But is this right, and if so which package ? I vaguely remembered this having been discussed before but I couldn't find the conclusions written down anywhere. I looked in quite a few places for answers to this: I searched the lintian tags for missing source, but they all seemed quite generic. I tried to search -devel archives for "bootstrap" (too many hits) and "bootstrap.min.js" (nothing relevant). I tried various wiki searches too. (The d/copyright problem with epydoc should be easy if tedious to fix; I don't understand why it wants epydoc which I thought was obsolete but this is far from my field of expertise.) Regards, Ian.
Re: Announcing miniDebConf Montreal 2020 -- August 6th to August 9th 2020
Zlatan Todoric writes ("Re: Announcing miniDebConf Montreal 2020 -- August 6th
to August 9th 2020"):
> So, could we avoid in future places that are politically very sensitive?
We should consider all the implications of our venue choices,
including political questions.
> And while I am at that, I will mention one more upcoming bid - Kosovo.
> It is part of Serbia, still officially by UN resolution[0],
This whole section of your message is tendentious Serbian imperialism
and very misleading, starting here. But it is also irrelevant to the
Montreal minidebconf so I don't feel the need to rebut it in detail in
this thread.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Announcing miniDebConf Montreal 2020 -- August 6th to August 9th 2020
Roberto C. Sánchez writes ("Re: Announcing miniDebConf Montreal 2020 -- August
6th to August 9th 2020"):
> On Sat, Feb 15, 2020 at 12:05:29AM -0500, Jerome Charaoui wrote:
> > Following the announcement of the DebConf20 location, our desire to
> > participate became incompatible with our commitment toward the Boycott,
> > Divestment and Sanctions (BDS) campaign launched by Palestinian civil
> > society in 2005. Hence, many active Montreal-based Debian developpers,
> > along with a number of other Debian developpers, have decided not to
> > travel to Israel in August 2020 for DebConf20.
>
> Would it be possible to not constantly air our personal political
> opinions and grievances on Debian lists? Especially as part of
> something that goes to an -announce list.
The choice of venue for Debconf is a political act. In this
particular case, it is highly political. It is political in a way
that the people who advocated for this venue, and those who chose this
venue, surely recognised.
Pretending that only the dissenting opinions are political, and then
asking for "political" opinions not to be aired, reframes the debate
in such a way that only the status quo can even be expressed.
It was IMO completely appropriate for the Montreal team to make their
position, and their actual motives, clear.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Accepted dgit-test-dummy 1.30 (source) into experimental
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Mon, 13 Jan 2020 11:38:21 + Source: dgit-test-dummy Binary: dgit-test-dummy Architecture: source Version: 1.30 Distribution: experimental Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: dgit-test-dummy - Dummy package for testing dgit infrastructure Changes: dgit-test-dummy (1.30) experimental; urgency=medium . * Test after cgi-grnet-01 upgrade. Checksums-Sha1: 2e780260ae02a075ee279f3dfffd1e9ab39b1059 1203 dgit-test-dummy_1.30.dsc 7ac909cae5f8cc139d1653a20f405616f12d198c 2037 dgit-test-dummy_1.30.tar.gz Checksums-Sha256: 903736b50f9dee0176e9c75722704ce19feb05c26630bac99160b73cc2c99f48 1203 dgit-test-dummy_1.30.dsc 13e3f519cf3d4126fe6a01455418afc783c980897c806833886f8e9e7b0e8756 2037 dgit-test-dummy_1.30.tar.gz Files: 5ef04b07713de2d01e60aab57ae369b9 1203 misc extra dgit-test-dummy_1.30.dsc ebbd3a45485fe36eb4cb9ed73e637adb 2037 misc extra dgit-test-dummy_1.30.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl4cVzsACgkQ4+M5I0i1 DTk8+ggAvVJi3nAnkTPNyJXaKZsLallYeLupYGXPIpPkjMsR5ej/9IdsW9bU1msL +XMN90r+4fRr/lEvD2tQzPcM90po7EG2EVh18rPqg60EVPXTx/s4kxgIWVukh8uk lt5eajGA+6t/09ET3B730h+8refOfTWI/w4uf5p3o91cJYpBHRM7HsO1FMJXWDL5 IoQPuIMULb7y7hHYYGPDYrb/5M5PMMSM62CNCKYLUBVpXhUS/EWUxR+Qh262RZ2k u6L3GNh01R2Tm4C743zXdYKV6oipSgPIpTQXB7n4VZdOLpSaQTh7weIVKDkwx48O 5GsrkLgHlhH4ev2kRbUNeAvzIHv1Kg== =g8QM -END PGP SIGNATURE-
Re: [debian] git depency on git-man
Anatoly Pugachev writes ("[debian] git depency on git-man"):
> Gerrit, hello.
>
> Is it possible to close bugs related to git/git-man and dependency
> between them?
>
> As I understand, fix should be quite trivial (just move git-man from
> git depends to recommends or suggests).
>
> debian bug ids (some started almost 10 years ago, some with patches attached):
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=613892
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=688717
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=933402
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948151
I think #613892 has a patch to do the --help compatibility thing, and
that plus the change to Recommends seems to me like it ought to be
uploaded ?
You could combine those changes into a single patch, send it to
613892, and tag the bugs "patch". That would increase the chances of
it appearing in the next upload.
If the maintainer is overworked, perhaps they would appreciate an NMU.
I haven't looked through the src:git bug list but perhaps there are
other bugs with patches that could be applied. I will leave
negotiating about that with the maintainer to you.
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: difficulty in understanding options in init-system-GR
Ian Jackson writes ("Re: difficulty in understanding options in
init-system-GR"):
> Mo Zhou writes ("difficulty in understanding options in init-system-GR"):
> > I went through the options in the init-system-GR[1] but I feel difficult
> > to tell the subtle differences between the options. I'd like to ask for
> > some hints here.
>
> I have written a blog posting about this:
>
> Debian init systems GR - voting guide
> https://diziet.dreamwidth.org/3999.html
>
> > In which way(s) are options (B, A, D, H, G) different from each other?
> > A couple of keywords should be helpful enough to differentiate them.
>
> I think my posting will answer your question. If not, please let me
> know...
After hearing of more than one person confused I wrote a followup
posting about the ballot paper format:
https://diziet.dreamwidth.org/4236.html
Maybe after all this I will have time and motivation to dust off my
work-in-progress patch series to make devotee be able refer to choices
with (only) letters, which I think would help a lot.
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: d/changelog and experimental
Overall I think it is usually best to include the changelog entries
for experimental versions in the appropriate place in the changelog
for the sid upload, unless they really contain only stuff that was
later undone. That is helpful for humans and also for computers.
For example, consider what ought to be shown by apt-listchanges
(which shows you the top part of the changelog since the version being
upgraded from).
Simon Richter writes ("Re: d/changelog and experimental"):
> Bugs are closed based on the changes file, which is generated from the
> topmost entry, always.
This is not strictly speaking correct.
If the most recent upload to sid (say) is not the second-topmost
entry in the changes file, then the .changes for the sid upload
should contain *all* the entries since the last upload to sid.
This can arise in situations other than where the intervening versions
were upload to experimental. Perhaps the intervening versions were
not uploaded at all, or were REJECTed, or were uploaded to another
distro. [1] In those situations the intervening bug close entries
need to be processed.
You can get this right by passing a -v option to dpkg-genchanges via
the appropriate mechanism for your usual build and upload tools.
Or you can just use dgit which gets this right automatically :-).
Ian.
[1] A workflow is possible where you use one git branch and just add a
new changelog entry for each upload. Although our data formats would
support uploads to a multiple distros with a single changelog entry
and even a single signed .changes, unfortunately our various tools and
services don't. But writing an additional changelog is easy.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: difficulty in understanding options in init-system-GR
Mo Zhou writes ("difficulty in understanding options in init-system-GR"):
> I went through the options in the init-system-GR[1] but I feel difficult
> to tell the subtle differences between the options. I'd like to ask for
> some hints here.
I have written a blog posting about this:
Debian init systems GR - voting guide
https://diziet.dreamwidth.org/3999.html
> In which way(s) are options (B, A, D, H, G) different from each other?
> A couple of keywords should be helpful enough to differentiate them.
I think my posting will answer your question. If not, please let me
know...
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Accepted pm-utils 1.4.1-19 (source) into unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Thu, 14 Nov 2019 15:08:18 + Source: pm-utils Binary: pm-utils Architecture: source Version: 1.4.1-19 Distribution: unstable Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: pm-utils - utilities and scripts for power management Closes: 852167 Changes: pm-utils (1.4.1-19) unstable; urgency=medium . * No-change upload to make myself the maintainer. I intend to perhaps share this with the DEIDT (Debian Ecosystem Init Diversity Team), but since it's O: right now and has an RC bug open, I want to get going. Closes:#852167. Checksums-Sha1: 25044be5188f342f80a2459bcb79765c4006391c 1636 pm-utils_1.4.1-19.dsc 29dee1b4a2697f1486c5a740d7f80e69e8b9a73a 27416 pm-utils_1.4.1-19.debian.tar.xz Checksums-Sha256: a2a0d9e9d99e3fe47c4d1e6e4bb117612462a54e0dcb2342700a290fdd6a0e06 1636 pm-utils_1.4.1-19.dsc aeff21b0e06edf5a4ed2f2363ab127dabf097c86207414b956d65b481ca2fa70 27416 pm-utils_1.4.1-19.debian.tar.xz Files: 882a36212bbec8f7881e5d8199aa6340 1636 admin optional pm-utils_1.4.1-19.dsc 422d16bef2e85b32dc157ea5a72fffd2 27416 admin optional pm-utils_1.4.1-19.debian.tar.xz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl3NfFIACgkQ4+M5I0i1 DTkaMwgApqxg1M+8H+d3zDiPSQU/bkpG6727ln/AyvJrEgTdMa5D2KYaHHQiSX+P si6oYdCUxMPNxxJPVkm/OgCIQ+NhJSQo/CTEpjGpqLLdpD2Qt4Dj+BRb5CcUeM2g V3OO2o1wbATdngwvwEosnq0n3F17UG/CABUpTJ5V0SaCWS+rSJpxjNjDiqkXm93z L0y932cTnHWuZ1eQ6J/tv89QiWKTZK2UqkURB5IkDFMdBU89s1cQDCr/hyvcOPOU E9VCamHKWHPF+PIvoH9K1STVu6r23fehz0i2QQg20tX/5m6P+tWZMHk/p0/0eBuf 9wMJuT8eikBptohIGVSia6O/2d37Qg== =ZPii -END PGP SIGNATURE-
Re: [RFC] Proposal for new source format
gregor herrmann writes ("Re: [RFC] Proposal for new source format"):
> On Thu, 31 Oct 2019 11:59:07 +, Ian Jackson wrote:
> > * tag2upload service, or some related service:
> > - determines that the maintainer is using a dgit-compatible git
> > workflow, by looking at the tags, and looks at some in-dsc
> > metadata to find the maintainer's repo
> > - determines that the maintainer is using salsa or launchpad,
> > converts the NMU to the maintainer branch's format, and
> > submits an MR
>
> … after checking that the current state of the git repo corresponds
> to the current version in the archive (which is often not the case in
> my experience with NMUs) …
Right. This is one of the difficulties with the ad-hoc maintainer
branches you find on salsa. And it is one of the ways that dgit
helps.
An NMU done with `dgit push-source', and starting from `dgit clone',
always gets this right. But if the maintainer didn't use dgit to
upload, `dgit clone' produces useless history, so the NMUer has to
cope with lack of history; or if the NMUer wants history they can dig
around in salsa. But if the NMUer fetches a branch from salsa it is
not so easy to make sure that they start from the right git commit
(and not possible to make a tool which gets it right every time).
My enhanced scheme as I propose above could also get all this right.
The NMUer would use `dgit clone' and they would see proper history
because the maintainer had used tag2upload. So the NMUer's
canonical-view git branch starts at the last upload, as it should.
The tags specify where all the relevant versions are. So the
converter can make a maintainer view branch starting at the
maintainer's last upload. That's what would become the maintainer
view MR. (It could be rebased if desired.)
Ian.
Re: Integration with systemd
Martin Steigerwald writes ("Re: Integration with systemd"):
> As to this, I did not yet see that the migration of elogind to testing
> has been accepted.
Yes.
I find these conversations draining, exhausting, awful. I am sure
that most people who are sceptical of systemd agree. The constant
negging and doom-saying is very unpleasant.
Meanwhile, elsewhere, where the real technical work is being done,
systems are being built that run modern software (including GNOME,
whatever the latest daemons, etc.) without systemd.
The question is: are we going to permit those technical contributions
into Debian ? Are we going to keep making it awkward or are we
actually going to _welcome_ them ?
Are we going to say to those of our contributors who want to see a
nice tidy hegemony, "sure, throw all the sysvinit stuff away, it is OK
to reject the patches with init scripts, it is OK to work to make it
impossible to have this stuff in Debian, even if the software works" ?
And, are we going to continue to wear people down with awful threads
like this one, where a parade of doomsayers tell us we can't have what
we want *even though it already exists and is maintained* ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: [RFC] Proposal for new source format
Sean Whitton writes ("Re: [RFC] Proposal for new source format"):
> Sorry, I didn't phrase my suggestion carefully. I was assuming that we
> will continue to expect maintainers to accept patches in the BTS, but
> that if they *prefer* something else, they could document that in
> README.source.
>
> Someone making a large number of changes could just choose to submit
> them all as patches to the BTS, due to the high cost of checking
> README.source -- I'm sure maintainers would understand this.
Well, that's fair enough as far as it goes. But I think we could do
better.
It would be possible to imagine some service that works like this:
* NMUer does dgit clone, makes changes, does tag2upload with some
option (ie a parseable note in the git tag) to say "automatically
do the NMU things"
* tag2upload service, or some related service:
- determines that the maintainer is using a dgit-compatible git
workflow, by looking at the tags, and looks at some in-dsc
metadata to find the maintainer's repo
- determines that the maintainer is using salsa or launchpad,
converts the NMU to the maintainer branch's format, and
submits an MR
- files a Debian bug referencing the MR
- if the preconditions are not satisfied, sends a traditional
debdiff by email to the BTS instead
* Maintainer looks at the MR. (If discussion is needed they do it in
the bug [1].) Maintainer merges the branch to master.
* git hosting service autocloses the MR. Metadata gateway service
marks the Debian bug pending.
I think this would give us most of the best of the various possible
worlds. (You could also do most of this without the actual upload of
course. Such an canonical-view-to-maintainer-MR gateway would already
be possible, and would work when the maintainer used dgit.)
Ian.
[1] IMO bugs are better for this because they provide a less bitty
conversational structure and are archived and published more usefully.
But it would be possible to handle this via the hosting system MR
conversation instead, or maybe mirror the conversation.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: [RFC] Proposal for new source format
Russ Allbery writes ("Re: [RFC] Proposal for new source format"):
> Ian Jackson writes:
> > Of course this means that the resulting source packages are not the "3.0
> > (quilt)" patch queue source packages that many people (even some people
> > who like git) say is important to them.
>
> > A key design goal for dgit and my tag2upload proposal, is that (when
> > used in the most usual way) it produces nice source packages like
> > everyone is used to.
>
> My recollection is that you found 3.0 (quilt) packages had a lot of edge
> cases and strange interactions with Git that you've had to work around.
Oh certainly. I don't like them very much. However, lots of people
have, over a long period, told me that they like them and that their
features are valuable to them. This comes up over and over again in
threads like this one.
> I think there may be some deep conflicts here between a source package
> that is inherently a useful basis for work and modification (one of the
> design goals of 3.0 (quilt), and also one of the things those of us who
> like Git source packages have always wanted) and a source package that is
> easy to reproducibly generate and contains as little complexity as
> possible so that the archive software doesn't need to use any complex
> tools.
My response to this situation has been to solve it with superior
technology. dgit is a reliable bidirectional (mostly [1]) converter
between .dscs including `3.0 (quilt)' and useful[1] git branches.
That is its core purpose.
I have certainly encountered a large number of anomalies and
difficulties but I have overcome them and the result is a system where
everyone gets to keep what they value.
I took this approach because I wanted to make new stuff that people
would *enjoy more than the old stuff* and *want to use*. Software
whose output everyone would like.
[1] If to you `useful' means patches-unapplied or bare Debian, then
the dgit ecosystem does not yet have a converter from dsc to your git
branch.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: MBF: don't build against libatlas3-base if possible
Mo Zhou writes ("MBF: don't build against libatlas3-base if possible"):
> Type: performance improvement, integration
> Affected: `apt rdepends libatlas3-base`
> Severity: important
Hi. I would like to congratulate you on your work so far. I am no
expert on this area but from what I read, I want to encourage you :-).
I wanted to comment on this MBF. If what you say is true, it seems
right in principle. However, your MBF proposal email was missing the
bug template.
I think it's important to give the MBF recipients a clear set of
instructions on what to change. And to assume that they may not know
what they are doing :-).
Your email gives clear instructions on what change to the .deb is
wanted:
> [...] Ideally any program that needs BLAS/CBLAS ABIs should link
> against libblas.so.3 instead of libcblas.so.3 (note the char "c" in
> middle)
But it is not clear to me how this should be achieve in a source
package which currently build-depends on libatlas3-dev. Can the
build-dependency simpy be swapped out ? If so, to what ? Or are
other changes typically needed ?
(Please forgive me for writing this mail without having actually
looked at any affected source package.)
> @Andreas, can we add this point to science-team policy?
I will leave this to Andreas. But if Andreas and the science-team
agree with you, then this is clearly a candidate for a lintian
warning. You probably want to file a bug against lintian.
Please keep up the good work.
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: [RFC] Proposal for new source format
Russ Allbery writes ("Re: [RFC] Proposal for new source format"):
> And therefore the goal of this proposal is to define a source package
> format that allows this to be done more easily than our current source
> package format allows?
Of course this means that the resulting source packages are not the
"3.0 (quilt)" patch queue source packages that many people (even some
people who like git) say is important to them.
A key design goal for dgit and my tag2upload proposal, is that (when
used in the most usual way) it produces nice source packages like
everyone is used to.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)
Helmut Grohne writes ("Re: Building Debian source packages reproducibly (was:
Re: [RFC] Proposal for new source format)"):
> I think I'd trust the tag2upload service given the documentation you
> presented about it. I'm less faithful in all dgit installations being
> sane, sorry. We've run into too many builds in dirty chroots already.
That does make sense. This is one of the ways that tag2upload is
better than dgit push. (It is a shame that "integrity" concerns are
blocking integrity improvements.)
It would be possible to write a QA service which would verify Dgit
fields and automatically file RC bugs. So far that hasn't seemed
necessary.
It would also be possible for dgit clone to verify the correspondence
itself, at the point where it honours the Dgit field. Would that be a
useful feature for you ? Of course it does mean downloading the
elements of the source package, which it currently doesn't need to do
if it finds a Dgit field, but there's no real difficulty. (I wouldn't
make this the default!)
> > You do not need to talk to any random git servers. The git tree is
> > available on a single official Debian server, the dgit git server.
> > The Dgit: field in the .dsc identifies the commitid. The .dsc is of
> > course available via the signed apt repositories, as well as being
> > available from the ftpmaster data API.
>
> I was not trying to imply dgit to be a random git server. Given that
> dgit (currently) only contains history for a fraction of packages, we
> still need to compare with Vcs-Git. Given enough time, dgit will have
> useful histories eventually.
Yes. If tag2upload is deployed, I expect it to be very popular.
Until then Vcs-Git has all the problems you mention and many others
too: it is hard to reliably find the right tag (even the tag name is
not formally standardised!) and certainly nothing checks that the tag
corresponds in any particular way. How it might correspond is
generally not even documented anywhere - at least, not anywhere
machine-readable.
> Hmm. I'm not sure whether I actually need the tag object. The commit id
> is what I really need. dak might need the tag object. I'll defer to
> others.
I think ftpmaster's concerns mean that dak would want the tag object
to redo the uploader identify verification, even though from my point
of view that would be a redundant check. But it's simple to provide
the tag and there is some integrity improvement from doing so, so that
is what I am proposing.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)
Helmut Grohne writes ("Re: Building Debian source packages reproducibly (was:
Re: [RFC] Proposal for new source format)"):
> In other words, I want these formats (source package and tagged git
> tree) to be isomorphic (minus history). This requirement is too strong
> since not every source package will have a corresponding tag, but when
> there is a tag, I want to safely go from source package to tag and back
> again and arrive where I started from.
I wonder if I have misunderstood you, because:
The tag2upload proposal is based on dgit, which already provides this.
dgit indeed defines an isomorphism between source packages and git
trees, and dgit clone gives a git branch that is thus-isomorphic to
the .dsc. This is fundamental to dgit's design.
With `dgit push', the isomorphism is checked on the maintainer's
machine during `dgit push'. With tag2upload it is ensured by the
tag2upload service. (When the uploader didn't use dgit, dgit clone
does a .dsc import, thus ensuring the isomorphism.)
> This property allows me to start from a git tree that is
> authenticated by dak rather than a random git tree on a random git
> server of questionable origin.
You do not need to talk to any random git servers. The git tree is
available on a single official Debian server, the dgit git server.
The Dgit: field in the .dsc identifies the commitid. The .dsc is of
course available via the signed apt repositories, as well as being
available from the ftpmaster data API.
It is true that this doesn't give you precisely the *tag* object -
just the commit. Adding the objectid of the tag object to the .dsc
Dgit: field would be easy, if that would be helpful to you. (Please
file a wishlist bug against dgit if so.) Alternatively, dak could
publish the tag object (in a similar way to how it publishes binary
buildinfos).
Note that there are *two* tag objects: 1. the canonical view:
the dgit view tag, which is simply-isomorphic to the source package.
2. the maintainer tag, which is on the maintainer's branch and refers
to a commit in maintainer branch format.
With dgit push these are both made during dgit push with the
maintainer's key. With tag2upload the canonical view tag is made by
the tag2upload service, because it is that service which performs the
maintainer->canonical conversion.
Each maintainer workflow defines a different mapping between
maintainer views and canonical views. The (currently supported[1])
workflows are all isomorphisms. So it is possible in principle to
reverse the maintainer->canonical transformation (if you know the
workflow, which can be found in the tags) but there is not currently
code to do that. I don't get the impression, however, that this is a
thing you feel you need ? (Some form of reverse transformation would
be needed to automatically and workflow-agnostically handle MRs whose
submitter is using the canonical view.)
> This backwards-connection seems to be missing thus far, but I do find it
> important for the reasons above. Adding it would easily allow dak to
> validate the signature on the tag.
So, I'm not sure I understand what you think is missing.
Ian.
[1] I think with monorepo workflows the maintainer->canonical
conversion is generally irreversible, because it discards information
about source packages other than the one in question. This wouldn't
block MR processing because MRs are deltas and by definition the other
parts of the monorepo aren't edited in the MR. It does mean you
couldn't reconstruct the whole monorepo given just the canonical view.
(Arguably this means that the .dsc representation of a source package
from a git monorepo is not a PFM. See arguments on -legal and
-project, passim. But the canonical view dgit branch does contain the
whole of the monorepo in its history, in a discoverable way, so
doesn't have this issue.)
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)
Didier 'OdyX' Raboud writes ("Building Debian source packages reproducibly
(was: Re: [RFC] Proposal for new source format)"):
> Where I'm coming from is that we were discussing the tag2upload
> problem at miniDebConf Vaumarcus. [...]
I appreciate your efforts to try to unstick all this.
> The hard part is not the packing and unpacking of the special tag; that's
> mostly just strings massaging. But building the exact same source package in
> different environments is harder than I expected.
Yes. I have code to do it for tag2upload, though. It's not released
yet because I stopped putting effort into this whole area after
getting discouraged.
> Of course, all of this can only work if we can have, or make the ".git to
> .dsc" conversion reproducible; hence my query.
>
> All-in-all; would this be a welcome mechanism?
I think by requiring the user to always have the tarballs on hand and
wait for them to be manipulated and maybe transferred, you are losing
a fair amount of the benefit of tag2upload.
But if you did want to do something along these lines, maybe you
should do it by adding code to git-debpush and the tag2upload service
rather than by reinventing the rest of the machinery ?
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Building Debian source packages reproducibly (was: Re: [RFC] Proposal for new source format)
Scott Kitterman writes ("Re: Building Debian source packages reproducibly (was:
Re: [RFC] Proposal for new source format)"):
> Effectively tag2upload would replace DAK as the entry point for
> packages into the archive (the equivalent to the current source
> package signature verification being the git tag signature
> verification). I don't think the discussion got to a point where a
> path forward that was considered reasonable by both the tag2upload
> developers and the FTP Masters was reached.
The current tag2upload proposal includes providing dak with the signed
git tag object so that it can re-verify the identity of the human DD
who authorised the upload.
The sticking point, as I understand it, is that this still does not
allow dak to verify that the *contents* of the .dsc were as intended
by the uploading human. [0]
In the tag2upload proposal, the conversion from git tag to dsc is
`merely' done by an official Debian service on an official Debian
machine, and is `merely' fully reproducible and auditable.
But this is not good enough for some ftpmasters, who want that
verification to be done *by dak*. Various people attempted in the
previous thread on this topic to find out *why* this is thought
important, without apparent success.
It would be nice to be able to work around this objection somehow by
writing more code. Unfortunately, any alternative - such as that
described by Didier earlier in this thread - has undesirable
properties. In particular, it does not seem that it would be possible
to do anything along these lines without continuing to burden the
developer's working system with a whole pile of messing about with
tarballs and quilt and so on.
For example, you would not be able to do this:
git clone salsa:something
cd something
make some straightforward change
git tag# } [1]
git push # }
Instead you would have to download the .origs and so on, and wait
while your machine crunched about unpacking and repacking tarballs,
applying patches, etc.
With tag2upload all that work is done for you on the tag2upload
service, which of course has a fast network connection - and you don't
need to wait for it.
Ian.
[0] Currently, of course, this requirement is not achieved for
existing git based uploads. In practice, DDs use ad-hoc
git-buildpackage runes on their local machine to convert git data into
.dscs. That conversion is not controlled, not reproducible, and not
practically auditable. I guess maybe those blocking tag2upload think
it is sufficient that we can blame the DD if it is messed up or
suborned ?
[1] In practice with tag2upload you would use `git-debpush' instead of
`git tag' + `git push' but this is a thin wrapper around `git tag' and
does not involve downloads or indeed any network activity, etc.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: [RFC] Proposal for new source format
Russ Allbery writes ("Re: [RFC] Proposal for new source format"):
> That said, Bastian's point about what we should do if we find that the Git
> repository contains something that isn't distributable is valid and needs
> to be dealt with regardless. I think one of our points of disagreement is
> that I don't see how this is a concern specific to the archive; we already
> have this problem because Salsa is an official project service, so we need
> to solve this problem for arbitrary Git repositories already.
It is also a problem that the dgit git service could face. That is
also an official project service. I have anticipated the potential
need to deal with this issue.
With the assistance of the dgit server admin, a maintainer can rewrite
the history; and the server admin can blocklist the troublesome git
objects (which will prevent them from ever being pushed again). The
server admin can also of course simply delete a package entirely.
So far this has not been necessary. I don't know how often a similar
situation has arisen with alioth and now salsa.
(I agree with everything else you wrote, too.)
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Debian and our frenemies of containers and userland repos
Paul Wise writes ("Re: Debian and our frenemies of containers and userland
repos"):
> On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote:
> > FYI, this is because autopkgtest has an abstraction for multiple
> > container/virtualization mechanisms (lxc, lxd, qemu, schroot)
>
> It seems like this abstraction should be split out of the autopkgtest
> source and then depended on by autopkgtest.
I agree. I was the original designer of this interface and the
original author of autopkgtest. I put the virtualisation abstractions
in src:autopkgtest just because it was convenient, not for any
principled reason. I chose the name pattern "adt-virt-*". I put
"adt" in it to avoid it being too much of a namespace grab, not
because this interface was intended only for the use of autopktest.
I think the current autopkgtest maintainer is not so keen on this
abstraction. I would love it if we could split it out from the
autopkgtest package and I would volunteer to maintain it.
Martin, would such a move have your support ?
> Do any other such abstraction layers exist?
The closest I was aware of at the time was libvirt, which is not
really useable in the same way. I think the interface I designed and
which has subsequently been significantly improved, is a good one, and
we should continue to develop and enhance it.
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Possible doc package side-effect from going source-only upload [and 1 more messages]
Dirk Eddelbuettel writes ("Re: Possible doc package side-effect from going
source-only upload [and 1 more messages]"):
> -build: build-arch
> +## edd 19 Sep 2019 for source uploads also build build-indep
> +build: build-arch build-indep
>
> build-arch: make-arch
> ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
>
> Fetched the package from pool and r-base-html is still 'empty'.
If you want to continue to try to fix this without a rewrite, I think
you need to repro locally. Have you tried the dpkg-buildpackage rune
which builds only arch:all, namely --build=all ? (NB that does not
mean build all the things, it means build only "all" things.)
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Possible doc package side-effect from going source-only upload [and 1 more messages]
Dirk Eddelbuettel writes ("Re: Possible doc package side-effect from going
source-only upload [and 1 more messages]"):
> On 19 September 2019 at 14:51, Ian Jackson wrote:
> | This would be a good idea. It is quite some effort but I think you
> | would be rewarded with significantly lower maintenance burden.
>
> Agreed in principle. Finding time to do it the right is the issue.
Indeed. Things that make this easier: 1. you'll get a good level of
support from the rest of the Debian community, who'll be generally
happy to help debug etc. 2. You will set out to produce identical
output packages, except for deliberate changes, so you can use debdiff
to see whether your new rules file works as desired.
Because of (2), you can just start with the trivial dh rules file and
iterate until it (i) builds (ii) produces the output you want.
I did this for src:xen (another complex package with a *lot* of, erm,
technical debt in its package build) last year. It took about 2-3
days' solid effort. I think it's been repaid already.
> On 19 September 2019 at 16:03, Guillem Jover wrote:
> | [a lot of detailed comments]
>
> [ Dirk's responses ]
In your position would definitely try to minimise the amount of work
to the existing rules file.
So for example:
> | - It seems generally parallel unsafe, as many targets declare what
> | should be serially executed as parallely-executed, as in:
...
> Fair. Do we build packages with 'make -j ...' now?
This can simply be disabled (and it's still disabled by default). I
would not encourage you to try to get rid of the concurrency bugs in
this rules file, other than by a complete rewrite.
Guillem's comments on this are of course very helpful, more generally.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Possible doc package side-effect from going source-only upload [and 1 more messages]
Dirk Eddelbuettel writes ("Possible doc package side-effect from going
source-only upload"):
> Maybe someone on the list can help with a sharp insight before I go trying.
>
> The r-base source package (for the R system and language) has a somewhat
> cobbled together debian/rules [1], mostly of my making over the last 20+
> years since I helped Doug more and more and eventually took it over. I
> apologize for the rough shape it is in, but hey, it works. Mostly. Read on.
> So presumably the dependency graph within debian/rules is wrong. Would
> anybody here know
> - either a failsafe idiom forcing the right thing to happen
> - or a more efficient way
> to ensure the binary-arch is built before binary-all? Should I force it? Is
> that wasteful? Is there a recommended way?
You could take the first part of the binary-arch target and split it
out into something that both binary-arch and binary-indep depend on.
That would probably "fix" this problem.
Holger Levsen writes ("Re: Possible doc package side-effect from going
source-only upload"):
> not really that helpful of a comment, but I think the rules file would
> be a lot more readable if you'd dropped all the old commented out code
> in it.
I agree with this.
> (and then I think^wbelieve your arch-all problem could be solved by
> switching to dh style...)
This would be a good idea. It is quite some effort but I think you
would be rewarded with significantly lower maintenance burden.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Accepted dgit-test-dummy 1.29 (source) into experimental
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Sun, 15 Sep 2019 21:38:20 +0100 Source: dgit-test-dummy Binary: dgit-test-dummy Architecture: source Version: 1.29 Distribution: experimental Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: dgit-test-dummy - Dummy package for testing dgit infrastructure Changes: dgit-test-dummy (1.29) experimental; urgency=medium . * Testing mirror rsync rune (after key puppet update). Checksums-Sha1: 865d257404672c14a0482abf5f5400c59c48e7bd 1203 dgit-test-dummy_1.29.dsc d3857e7b8a9ec707959548ac4499b201fbac6a7c 2015 dgit-test-dummy_1.29.tar.gz Checksums-Sha256: 2af6bebccfe62ae784850f857c59cd530c301649063717208fe1932c89ed020e 1203 dgit-test-dummy_1.29.dsc 0be9f4c236826c9ae841dcde7c752242e16bd2701947300967560412a6713e1e 2015 dgit-test-dummy_1.29.tar.gz Files: 0e58fc988a2b851567f7dfe7196dde97 1203 misc extra dgit-test-dummy_1.29.dsc 8b10811e3237675e01c628151150f723 2015 misc extra dgit-test-dummy_1.29.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl1+oXIACgkQ4+M5I0i1 DTkQEwf+KGKDwCb0vDVQld15vJA87MPJTzdHyX5asWGGOLOY9CJoz30K9FVghHeO HPmPG3XB8/9W/pORT4Gqwqq/6kMgRYKoX0DuOiHEvJmF5kLRwAq/7Gvc94B0883L cXzO9LvrRkNDQB1K6rTN9GvAo2RAYrP8LtquGe74BPt7anui8Oe50/XmZoeAvVP/ az3Wf+FxicoXRBpYrkN4sqVGgZSQo2Lg7lhYVdlANDRmdh05rKedgVPzA2WSCbEJ NW99edLFjskymcgTQENt91AR9Y9tpk8PyDWJUm1HuHpYpPBI7fz+Yi6FJn9rvqWt +Tu1sKB5qa2ctb8pOenjFt6PyyB8WQ== =w9S3 -END PGP SIGNATURE-
Accepted dgit 9.9 (source) into unstable
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Format: 1.8
Date: Sat, 14 Sep 2019 21:34:13 +0100
Source: dgit
Binary: dgit git-debrebase git-debpush dgit-infrastructure
Architecture: source
Version: 9.9
Distribution: unstable
Urgency: medium
Maintainer: Ian Jackson
Changed-By: Ian Jackson
Description:
dgit - git interoperability with the Debian archive
dgit-infrastructure - dgit server backend infrastructure
git-debpush - client script for git pushing to Debian-style archives
git-debrebase - rebasing git workflow tool for Debian packaging
Closes: 935084 935443 939679
Changes:
dgit (9.9) unstable; urgency=medium
.
Documentation:
* dgit-maint-{merge,debrebase}(7): Suggest a better rune.
(`git fetch --tags upstream') Closes:#939679. [Sean Whitton]
* git-debpush(1) minor improvements. Closes:#935084. [Sean Whitton]
* dgit(1): Fix a typo (found by lintian)
* dgit-maint-bpo(7): Mention occasional need for --new. Closes:#935443.
* git-debrebase: Extend extended description for this .deb.
.
Infrastructure:
* New script dgit-mirror-ssh-wrap suitable for use as restricted
command on repos mirror target hosts, and corresponding test.
.
Packaging niggles:
* control: Add missing dependency on liburl-perl (dgit and tests)
* control: Drop redundant Priority field for dgit-infrastruture
* control: update Vcs-Git to end in .git
* lintian: Override some checks
* control: Bump Standards-Version to 4.4.0.0.
Checksums-Sha1:
d10934d0eecc319e8388abf73c1e18b0bf4cd1c1 1841 dgit_9.9.dsc
8883c9c6e09b7d5ee06ad699729fe983aacee49a 697096 dgit_9.9.tar.gz
Checksums-Sha256:
34640af4dcf885d99295473a634f35c5479f4dc74d146c8be34b8c507f4e 1841
dgit_9.9.dsc
abf52a9c1c7b92986178f4c12fb2911d2a0a9e085fbdce0ea0a9d199a6a3ea42 697096
dgit_9.9.tar.gz
Files:
764baea302971700a20e7197acce5f94 1841 devel optional dgit_9.9.dsc
02e9dfecea90ad450242200c6e3a8c70 697096 devel optional dgit_9.9.tar.gz
-BEGIN PGP SIGNATURE-
iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl19T1wACgkQ4+M5I0i1
DTkBSAgAnliONklayC2f59WebLdR4wvLGAY76mqMPAegMk1gKik+K98Zb1v8zCwm
JLlU+HgP/PiymlBoufTYLgiboFHBhd7CJqiYPE1EHAY9HouljIOSpCeIj0uQ7E/4
4W8Bzn5FAwvvGmxVKR3PaKCclEEqjFasF5EojztM0Xbxy1VSGCHYMmtypAM31E48
v+g9sEyh1IqAD4mVOKquAF3ikblGSqkoWvdRLjH2IYbfCrA6R0RjqN42uijy9kj1
7QV9IydTfx4g4I/aQqGaN0iJetwUFRdzLpxWBhQAAYoZz+gGt1UouMAvXZj5d6hh
9tEbfKO9BGMNTaTYENPhNONhK1eY+A==
=+LWL
-END PGP SIGNATURE-
Accepted dgit-test-dummy 1.28 (source) into experimental
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Sat, 14 Sep 2019 19:57:08 +0100 Source: dgit-test-dummy Binary: dgit-test-dummy Architecture: source Version: 1.28 Distribution: experimental Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: dgit-test-dummy - Dummy package for testing dgit infrastructure Changes: dgit-test-dummy (1.28) experimental; urgency=medium . * Testing mirror rsync rune (after key update). . dgit-test-dummy (1.27) experimental; urgency=medium . * Testing mirror rsync rune (after dgit-live update on cgi) . dgit-test-dummy (1.26) experimental; urgency=medium . * Testing mirror rsync rune. Checksums-Sha1: b6d6d75116298d0c439949382a3f81e9c51d1aee 1203 dgit-test-dummy_1.28.dsc fcce21c6c6e3851693e872b3e1c3748af36f3e79 1998 dgit-test-dummy_1.28.tar.gz Checksums-Sha256: 8a49edf811ab726c1e0572202c2c84b88be8fcc2f8621fc3ad5e94799fceb760 1203 dgit-test-dummy_1.28.dsc 6becbe8394961b24ba17f4d842ee4300e3e0ba52860a29f85027a5593a83e24c 1998 dgit-test-dummy_1.28.tar.gz Files: 5935463049348c8d76c6ae060d6a04f5 1203 misc extra dgit-test-dummy_1.28.dsc 4027dcc6b62cc32e839c963b851aa112 1998 misc extra dgit-test-dummy_1.28.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl19OFUACgkQ4+M5I0i1 DTlP8Af/XJ8AzIQo/g+iL35WC343QwLswLYI8iRBciPOc8fsDVmmfKQ31i5SBbU3 HEC3+zVC+jFFHRdTP7n4KzDW09u8EPJrpa2sTVzuSdQTDFgf06XqQne7uTDgxICZ RUJV8eRgbPAPFWQ8IfkJ+yQqdFsdWj9XLXx/pB47yH+HM8ai+K5DA7rPGNZcRWMe TQsW+Agc4OqDQVQeI80TE4J32HzM+LsLBsBp5YIJlVQS3C90aaJqzFpp9/bXF4rD qcM8To64jTHpTGLPlsm/MrxOxEc2C1wxjuvJNz416jUBXxnUzjoFtG93eI8S4Z+K 0uVV4r692u3JpJIBJrRs+CxjwUNgTw== =tvyb -END PGP SIGNATURE-
Accepted dgit-test-dummy 1.27 (source) into experimental
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Sat, 14 Sep 2019 19:50:07 +0100 Source: dgit-test-dummy Binary: dgit-test-dummy Architecture: source Version: 1.27 Distribution: experimental Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: dgit-test-dummy - Dummy package for testing dgit infrastructure Changes: dgit-test-dummy (1.27) experimental; urgency=medium . * Testing mirror rsync rune (after dgit-live update on cgi) . dgit-test-dummy (1.26) experimental; urgency=medium . * Testing mirror rsync rune. Checksums-Sha1: ac24042db527884a7c8649c5401375a248a1544c 1203 dgit-test-dummy_1.27.dsc c580ae8f0637c6fe6299c47b019b7ae4ad639200 1955 dgit-test-dummy_1.27.tar.gz Checksums-Sha256: f15776a37bdcc01c040533916d9f6ffeda7809391bdc1d0e863582be4ae76222 1203 dgit-test-dummy_1.27.dsc a059ec3fb2d190438be704300a3723da33cd78764b6f5ea745b9915dc7c47135 1955 dgit-test-dummy_1.27.tar.gz Files: ad1ee837208e5b9303aea140c4fc38c1 1203 misc extra dgit-test-dummy_1.27.dsc dfc2af1cab1c151dc22a850823ca1c1c 1955 misc extra dgit-test-dummy_1.27.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl19NocACgkQ4+M5I0i1 DTkJBgf/WzDpGQVw5RR/gQn6RV6QeaY32qj6caFHGG/VkK+mQ9pRVRTGNsV90Zs0 WEhA3mDiFhFhm6+gAUZDLcWddRfsYgSx2xHP2Nub1GJBz2ZFq9VuTZ1QvTFT2XEF aO70vKdH6h/dYhir7L/075y6Z57cCM/n0CWXuhs5uNbE6hg1g9rDEAW6G3ZE1Erz Q70DJDrX0AwC83I9v9MUzF7pZ+Ypc+CvhCtNFFCFpHf/w9KUdwOtjF5RCgG1hAV2 IHlJQRPIG4NrnIq2bWGzynHy+RvjbuYvbjQkBp+qLyuDYvSDIE9aedicoKBn3M5s TaW3iDe1p2TnfaycDQPpDL9eRrXzwQ== =i+QG -END PGP SIGNATURE-
Accepted dgit-test-dummy 1.26 (source) into experimental
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Sat, 14 Sep 2019 19:45:36 +0100 Source: dgit-test-dummy Binary: dgit-test-dummy Architecture: source Version: 1.26 Distribution: experimental Urgency: medium Maintainer: Ian Jackson Changed-By: Ian Jackson Description: dgit-test-dummy - Dummy package for testing dgit infrastructure Changes: dgit-test-dummy (1.26) experimental; urgency=medium . * Testing mirror rsync rune. Checksums-Sha1: 88989de47d77573330c4af714cf9783068bc103b 1203 dgit-test-dummy_1.26.dsc ed0a236bc2154f95326af0454207212f05c64498 1931 dgit-test-dummy_1.26.tar.gz Checksums-Sha256: b317be6a6bdffd23f4e7608228bf65ade0bd6d71fbe36b65a4f0990b9b0ae4c2 1203 dgit-test-dummy_1.26.dsc 9c7ffcbb9cf4c7080c7f0a41d5410d61e0dd07fc9248f18b6d2ebc609df594dd 1931 dgit-test-dummy_1.26.tar.gz Files: 1f728ec67d565e9f7909a463b73ae541 1203 misc extra dgit-test-dummy_1.26.dsc 97dd3c15b5c82c47d16b1b86df28a32a 1931 misc extra dgit-test-dummy_1.26.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl19NjkACgkQ4+M5I0i1 DTkfVAgAw8kT4jFFSr3z1Pz6/4UTUOkyhhnvb10akMqeJ5bkwyslNlGaA2ENEooF Ot2LSarSvGhmbWgcDU0VpwEr+i4OhRz+Ht+dR10e+xFBewzvlYV4J3g/ND8UMW48 02Ak10CS+txAFLrtI0nXLs6vxsY8ZSBYz9cQIUrw7wwd3Ta9EMkgueHNv941BDYF CNDJrGsZmzDJg/2a27jpdwCdu3mcCPTCvFsO8mqXaUBOoOfaIU8mX/zbi/NX8gia kjPULvAM5LDopgbqjctGynDVnqGYjgdX8GkyXU5VHKpCJorkzRnzCJQT0cXbggG0 QqxU1enU8251cNeaunAogxw1EL8UPg== =ezd8 -END PGP SIGNATURE-
Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github
Sam Hartman writes ("Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github"):
> Ian Jackson writes:
>
> > Sam Hartman writes ("Re: Git Packaging Round 2: SHOULD Not or
> > MUSt NOT Github"):
> >> Unfortunately, I believe you are in the [wrong] when judging
>
> Ian placed the word "wrong" in my mouth replacing the word "rough" from
> my original mail. I disagree with that characterization of what I said
> in the strongest possible terms.
I'm sorry, I genuinely thought you had made a typo. The sentence
Unfortunately, I believe you are in the rough when judging rough
consensus on this issue.
didn't make sense to me. I should have stared at it for longer and
then maybe I would have seen what you meant.
>When I say that Ian is in the rough, I mean that I think there is
> not a perfect consensus on the issue, and that Ian holds an view
> that is inconsistent with what consensus exists.
Thanks for the explanation.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: should Debian add itself to https://python3statement.org ?
Jim Popovitch writes ("Re: should Debian add itself to
https://python3statement.org ?"):
> On Thu, 2019-09-12 at 16:01 +0100, Ian Jackson wrote:
> > Drew Parsons writes ("should Debian add itself to
> > https://python3statement.org ?"):
> > > https://python3statement.org/ is a site documenting the projects which
> > > are supporting the policy of dropping Python2 to keep Python3 only.
> >
> > That statement is a *pledge* to drop support for python2 by the end of
> > 2020.
>
> FWIW, that proposed ending date is 2020-01-01, ~110 days from now.
It says
| the following projects have pledged to drop support for Python 2.7
| no later than 2020, coinciding with the Python development team's
| timeline for dropping support for Python 2.7.
which is rather ambiguous.
If we do interpret it to mean 2020-01-01, I doubt there is any
realistic chance of us making that, even if we decide we want to.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: should Debian add itself to https://python3statement.org ?
Drew Parsons writes ("should Debian add itself to https://python3statement.org
?"):
> https://python3statement.org/ is a site documenting the projects which
> are supporting the policy of dropping Python2 to keep Python3 only.
That statement is a *pledge* to drop support for python2 by the end of
2020. Have we in fact made such a pledge ? I think I may have missed
the memo that python2 would be removed from bullseye.
I did some searching and found this
https://lists.debian.org/debian-python/2019/07/msg00080.html
which is a sane-looking transition plan but doesn't seem to have a
timeframe and doesn't seem to contemplate removal of the actual
python2 interpreter.
FTAOD I don't have an opinion about whether bullseye *should* ship
without python2. Obviously dropping it would not be desirable from
users' pov, but maintaining an ancient thing by ourselves would not be
desirable either. I think I trust the Debian Python team to make that
tradeoff.
But we need to be clear what's going on and communicate early. If
python2 is going out of bullseye then there are a lot of bugs that
should probably be marked rc fairly soon...
thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github
Enrico Zini writes ("Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github"):
> On Thu, Sep 12, 2019 at 02:07:52PM +0100, Ian Jackson wrote:
> > I think this does not demonstrate that I am wrong about project's
> > overall opinion about this. I am confident that a GR to forbid this
> > would succeed.
>
> For what is worth, I would vote against such a GR.
> I'm extremely uncomfortable reading what you wrote.
I'm sorry to hear that.
> I see you keep pushing things with a strong cohercive slant rather
> than working on creating useful and attractive infrastructure to
> make everyone's work easier.
The latter is what I am trying to do. I'm sorry that the opposite is
occuring.
> I wish this work would be grounded instead on an acknowledgement and
> acceptance of the, imperfect, diverse, yet still valid status quo.
For me, my opposition to github has nothing to do with my desire to
improve Debian's workflows.
I am indeed trying to improve Debian's workflows by providing better
options. Options that I hope people will voluntarily adopt, and that
will become more officially recommended - but *not* mandatory.
On the other hand, my opposition to github is like my opposition to
the inclusion of software in main which automatically and without
adequate user permission downloads and runs proprietary binary DRM
code. Or like the arguments we've had over the lack of proper source
code for some javascript and machine language programs. Software has
been blocked from Debian, and valuable contributors discouraged, as a
result.
Should we also tolerate these freedom problems as an "imperfect,
diverse, yet still valid status quo" ? Is it unjustifiably "coercive"
to block non-free software from Debian main ?
I guess one lesson I should perhaps learn is that it is difficult for
me in particular to push on these kind of software freedom issues when
they are entangled with workflow issues, because of inevitable
confusion/conflation/whatever.
So maybe I should leave the "Free Software Needs Free Tools"[1]
advocacy to others. I do still think it's important.
[1] https://mako.cc/writing/hill-free_tools.html
> Thankfully I still consider it to be so, with the exception of the
> occasional frightening cohercive twist in some of your mails.
Well, thanks for the rebuke. I hope I have clarified my thinking and
please do the same again in future. (Or, indeed, right now, if you
think this message is still frightening...)
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging Round 2: When to Salsa
Ansgar writes ("Re: Git Packaging Round 2: When to Salsa"):
> If dgit provides a program to figure this out, people interested in
> obtaining the information automatically can just extract and use that.
It is not possible to figure out the branch format automatically
given just the maintainer branch.
> (Using dgit to upload packages is sadly incompatible with best
> practices around packaging.)
Using dgit to upload packages is best practice.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github
Sam Hartman writes ("Re: Git Packaging Round 2: SHOULD Not or MUSt NOT Github"):
> Unfortunately, I believe you are in the [wrong] when judging rough
> consensus on this issue.
>
> This was discussed fairly recently on debian-project; my take is that
> Thomas Goirand represented a position roughly the same as your own.
> My reading of that discussion is that:
Thomas was making a lot of much stronger assertions about what should
be done.
> More over your claim that this is not our current practice runs counter
> to facts. Of the 26,480 packages in my unstable sources with a vcs-git,
> 1836 are on github. 7% seems much more consistent to me with "NOT
> Recommended" than "forbidden."
Blimey. I didn't realise that.
I think this does not demonstrate that I am wrong about project's
overall opinion about this. I am confident that a GR to forbid this
would succeed.
It just demonstrates that we have few working enforcement mechanisms
against contributors who violate our norms.
> Even if there is not rough consensus to forbid non-free services, I'd
> welcome help documenting the concerns that can come up.
I think this is a question of Debian's core values.
Given the current situation, with 7% of packages in violation of what
I see as a key norm, it seems that this cannot be resolved via a
consensus process.
We should resolve this with a GR. Something like:
Subject: Free Software Needs Free Tools
No Debian contributor should be expected or encouraged, when working
to improve Debian, to use non-free tools. This includes proprietary
web services. We will ensure this, insofar as it is within Debian's
collective control.
For example, Vcs-Git fields in source packages must not refer to
proprietary git code management systems. Non-Debian services are
acceptable here so long as they are principally Free Software.
We encourage all our upstreams to use Free/Libre tools.
We recognise that metadata in Debian which describes the behaviour
of those outside our community, for example fields which refer to
upstream source management systems, may (in order to be accurate)
still need to refer to proprietary systems.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Common configuration of Debianish build etc. tools
Hi. Our rich ecosystem of packaging tools has a large number of configuration options. In some cases, different tools need to have the same option set in the same way. But we have no common place to put this information and often not even common terminology. One example is: export-dir (gbp) buildresult (pbuilder) build-products-dir (dgit) build-dir (sbuild) When one of these tools invokes others, it can sometimes pass down an option, but unless we are to have a kind of universal wrapper program, this is not always possible. (And we have too many wrappers already.) So, it would be good for there to be a single way to configure these things. Every tool ought to look in this single place as well as its own configuration. It shouldn't just be an environment variable because that clutters the environment unnecessarily. I think it should probably be a config file in ~, overrideable with an environment variable. (Or perhaps the config filename could be overrideable with an environment variable.) What do people think ? If this is a good idea then we need to agree on a filename and format. The format must be easy to parse in a variety of programming languages including, in particular, shell scripts. Any suggestions ? Also there should be a registry document in some package somewhere, which declares the format and lists the known config keys and their semantics. Thanks, Ian. PS Note that we are talking here about settings which relate to personal preferences, or machine-local setup. Not ones which are properties of the package and which must therefore be done the same way by all of a package's maintainers - that latter information must be elsewhere (in the source package, recorded in salsa, or something.) -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Git Packaging Round 2: When to Salsa
y way. > Discussion Comments > --- > > The biggest question here is whether we have consensus that people > should be using Git. I think it is clear that we have consensus that if you don't know better, you should be using git. > 2) Upstreams that use some other VCS than Git. I need more input here. > Would we prefer that people have Debian only packaging repos? Would we > prefer that they use athe other VCS? Do we not have enough consensus to > have a recommendation in this case. I don't think we have consensus. The options are poor: - Use a non-git vcs for everything - Use mixed vcs (Debian packaging-only git, upstream in other vcs) - Use an other-vcs-to-git importing/tracking tool. > Account Transitions > === ... > There's been enough discussion of this issue that it seems like someone > should take on the task of trying to figure out what we as a project > want. Solutions might include: > > * Procedures that get followed when accounts are closed. We probably > don't want more effort added to the tasks of NM front desk, MIA or > DAM. But if additional volunteers stepped forward who wanted to help > with these transitions, it might make sense to approach things that > way. > > * Changing how Salsa and Debian LDAP interact. This would involve > figuring out what we want and working with the Salsa admins and DSA. Another possiblity is: * Accept that maintainer git repositories may move from one location to another, as a consequence of changes of maintainer, changes of maintainer status, or changes of maintainer preferences. This would be a lot less annoying if the information about where to find the maintainer's git repository were more easily updated. Consider how long it is taking to update Vcs-Git. Note that if uploads are done with dgit, the actual git history (including the maintainer's history) is in a systematic, discoverable place, on Debian core infrastructure, and maintained with the same access control as the archive. I guess you must know that I think we should recommend that people who are just starting out should upload with "dgit push[-source]". > Closing Thoughts > > > A couple of issues have been brought up where people objected to the > project recommending Salsa. > > I've considered the following issues: > > * Some have objected that Salsa is not sufficiently free. My > understanding of the argument is that Salsa is running a community > edition of Gitlab that is generally free, but contains non-DFSG free > components that we would strip/do strip from the package. The Salsa maintainers have made it clear that they do not intend to accept patches from Debian contributors. For me that means that we are not treating Salsa as free software (!) I have also heard disturbing rumours about exactly what we are taking from Gitlab upstream. In particular, I would like the Salsa admins to confirm that what we are running is something we built from source. > * Others have objected that we don't run the Gitlab package on Salsa. > I'll note that many core services do not run packaged code. I think this objection is unsustainable for the reason you give. In fact, not running packaged code makes it a lot easier for a service owner to run the version they want, of their service, because they do not need root privilege to install a .deb. > I've considered these issues, and my reading of the discussion is that > the community is aware of them, but they do not rise to a level where > they would challenge recommending Salsa. > > If you'd like to work on Salsa, including helping Salsa be more free, > reach out to the Salsa admins and see if they can use your help. I think that these objections do not rise to the level that we shouldn't recommend Salsa to people who don't already have an opinion. Ian. -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Accepted dgit 9.8 (source) into unstable
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Format: 1.8
Date: Fri, 06 Sep 2019 23:42:15 +0100
Source: dgit
Binary: dgit git-debrebase git-debpush dgit-infrastructure
Architecture: source
Version: 9.8
Distribution: unstable
Urgency: medium
Maintainer: Ian Jackson
Changed-By: Ian Jackson
Description:
dgit - git interoperability with the Debian archive
dgit-infrastructure - dgit server backend infrastructure
git-debpush - client script for git pushing to Debian-style archives
git-debrebase - rebasing git workflow tool for Debian packaging
Closes: 931212 934807 935802 935874 939280 939504 939564
Changes:
dgit (9.8) unstable; urgency=medium
.
dgit, important bugfixes for bugs introduced in 9.6:
* Once again cope with archive skew. Closes:#935874.
[Initial report from Colin Watson]
* Follow http[s] redirects again. Closes:#939564.
.
dgit, useful new behaviours and fixes:
* Use distro-info-data to resolve unknown suites. Closes:#931212.
[Suggestion from Simon McVittie]
* Try to warn about misplacement of dgit options. Closes:#934807.
[Useability report from Félix Sipma]
* Fail early when the archive/ tag already exists, ie when version
needs to be not reused. Closes:#935802.
[Useability report from IOhannes m zmölnig]
* Do not mistakely reject .asc and .sig files with push-source -C.
Closes:#939280. [Report from Simon McVittie]
.
docs:
* dgit-maint-{merge,debrebase}(7): Suggest git fetch --all --tags
as it handles tags better. [Sean Whitton] Closes:#939504.
* git-debrebase(5): Tiny typo fix.
* dgit(1): Some semantic linefeeds (no change to rendered output)
.
Supporting changes:
* test suite: New test for http mirror and archive skew
* test suite: Test already-tagged check
* test suite: Add --force-reusing-version to many tests as now needed
* Dgit: Minor refactoring of a regexp.
Checksums-Sha1:
9e04a99e734a0e23dd5a8d9a8c90c3434019b1da 1821 dgit_9.8.dsc
8ddb3136e6a59dbdc89a742c0644ab530d9f1d92 695751 dgit_9.8.tar.gz
Checksums-Sha256:
3166b8e7f5b6b99d729c2b491c616b3e4a420e0e5420fbb96df43373db1ad36a 1821
dgit_9.8.dsc
e85dd18b28c6c8dbb355c8d21312639810917f60e5d53c35fb1ec6e44072c2c9 695751
dgit_9.8.tar.gz
Files:
f78e30a02fb25d31d3652018a0aa3c8f 1821 devel optional dgit_9.8.dsc
692ed56f1a2a809781a46c367ae8b402 695751 devel optional dgit_9.8.tar.gz
-BEGIN PGP SIGNATURE-
iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl1y5K8ACgkQ4+M5I0i1
DTlj0Qf9E/SGIIkN+OBs6EFtkd+9NhVRKqPdW6igCDFstDcMMqA5cSPjzYXVHr+M
FOFshta6NVeCqlxLTyUZQ272hWrJtncBbZm648QfQQF3HWbXG3GHs7BhENL35FTB
2aOm0q6/UHTBv7nZoc5Sl409fd9ak8Z1KiDSLk8B5T8xH431s23GsqX8f+DYGh+F
ZFGtESZ0M67nLmVFnhk8dJfBFarVPcY2HfGnf/LYLKkZCXFoOYUtnJ8kRiKsMXQR
DHGncnd3MeBUHCaUajPsb03qfU2NtTYj6oFT5evSUCAc5uVbxyDvwR5KDxz5MYag
1vo8+Fa3H3AEBqb4ifHzCX/ilQqt8w==
=Amue
-END PGP SIGNATURE-
Re: On the Removal of src:tensorflow [and 1 more messages]
Mo Zhou writes ("Re: On the Removal of src:tensorflow [and 1 more messages]"):
> Thanks for the offering. I'm not eager to write such a wiki page
> because it's not urgent. Plus, currently I have a rather complete
> image on the "Future Debian and Artificial Intelligence" topic
> based on experience. Of course I won't be satisfied by
> only documenting these small points.
OK, fair enough. How about this
> I've already planned to write a long article on the whole topic
> with a good overview and some important details[1] including
> those mentioned in the previous mails. Please look forward to it.
> After finishing the article, porting it to debian wiki would be
> trivial enough.
... when you have done this please let me know and I will make a wiki
page at least referencing your document(s), and maybe summarisig.
If you could arrange to mirror your article on some Debian server that
would avoid possible future linkrot (although if you plan for your own
article to have a good longterm stable url then maybe that's not
needed).
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: On the Removal of src:tensorflow [and 1 more messages]
Jonas Smedegaard writes ("Re: On the Removal of src:tensorflow"):
> Please beware that there's a huge difference between being able to
> stumble upon your notes in a web search - and in the case of a wiki page
> to add additional notes to it - and then to have an open invitation to
> ask questions to an expert.
>
> I encourage you to help get your valuable information onto our wiki,
> regardless of the lack of feedback you have received on it.
Seconded. However, I don't think this...
> Our wiki does not use mediawiki but an older(?) markup called Creole.
Mo Zhou writes ("Re: On the Removal of src:tensorflow"):
> I dislike the mediawiki markup. [...]
... is likely to be an answer to that.
Mo: if you write up your notes in some plain text format I will put
them on the wiki for you. You should probably do that by replying to
this thread because of the principle of doing things in public.
Please be sure to CC me on the email. NB that do *not* intend to take
on the role of ongoing document editor and won't be incorporating
comments made in response to your mail.
Thanks,
Ian.
Re: Proposed build profile: noinsttests
Simon McVittie writes ("Proposed build profile: noinsttests"):
> I would like to propose this build profile for addition to
> . I'll add it to the wiki if
> there seems to be rough consensus.
>
> Name: noinsttests
> Changes content of binary packages: No ("C: N" on the wiki)
> Changes set of binary packages: Yes ("S: Y" on the wiki)
Your reasoning and conclusios make sense to me.
(FTAOD, I didn't reread the background materials but I found your
message admirably clear without needing to do that.)
Regards,
Ian.
Re: Git Packaging: Native source formats
Russ Allbery writes ("Re: Git Packaging: Native source formats"):
> [context: git-debrebase, git-dpm]
>
> However, while analyzing a rebased branch isn't as hard for other
> people as a branch with a complex merge history, it does mean that
> upstream has to find a way to extract patches [...]
>
> [context: git-debcherry]
> This lets you generate the patches for people on demand, but they aren't
> just sitting out there for anyone to look at whenever they want.
Yes.
> Anyway, this is probably only applicable to a minority of upstreams and
> packages, and more upstreams these days would just like all patches as
> PRs.
One way to look at all this is that fishing the patches out of
Debian's (or whatever other downstream's) processes into a
standardised form suitable for an upstream branch is work.
Nowadays most upstreams expect us to do that work. (The same way we
expect our downstreams to file bugs, not expect us to guddle around in
their systems looking for stuff to take.)
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
gregor herrmann writes ("Re: Git Packaging: Native source formats"):
> On Thu, 29 Aug 2019 22:03:18 -0400, Theodore Y. Ts'o wrote:
> > The problem I have is that "dgit gbp" doesn't extract the upstream
> > .asc.
>
> This sounds like #872864 in git-buildpackage.
Oh, interesting. In that case Ted might find that using `dgit
push-source', `dgit sbuild' or `dgit pbuilder' work where `dgit
gbp-build' doesn't. That probably involves manually running
pristine-tar(1) (maybe origtargz would do it the right way?)
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Russ Allbery writes ("Re: Git Packaging: Native source formats"):
> [ discussion of benefits of maintaining the Debian delta as
> a linear series of broken-down changes ]
>
> There are obviously ways to represent this with a pure Git repository, but
> apart from using a patch system on top of 3.0 (native), at which point I
> don't understand why one wouldn't just use 3.0 (quilt), they require
> multiple branches and thus aren't available directly in the archive.
This is not true. There are at least two ways of doing this without
using a patch system: git-debrebase and git-dpm.
Both of these use only a single primary git branch which contains both
upstream history, and Debian changes to upstream files represented as
git commits.
I can't speak for git-dpm, but with git-debrebase and 1.0 native
source format, there would not any patch files.
> Extracting specific changes by comparing only two Git branches with a
> complex merge history is certainly possible to do with native Git tools,
> but I would classify it as an advanced Git skill. I think there are a lot
> of upstreams using Git for whom that operation would still be quite
> challenging.
This is also not that hard, in simple cases. There is a tool
git-debcherry which can do it automatically. I haven't used it but
AIUI if your Debian delta queue has few commits, and doesn't have
commits which involve merge conflicts with upstream merges (basically,
if each change is carried Debian only for a short time), it will
always produce the nice output you would hope for.
> This is one of those cases where knowing your upstream is invaluable.
I certainly agree with this. I don't think anyone is saying that
using (say) a merging git workflow with a native source package format
should be universal, or even the default.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats [and 1 more messages]
Philipp Kern writes ("Re: Git Packaging: Native source formats"):
> While this may be true on some level, it is also important to be able to
> build packages from checked-out source trees (say, git repositories)
> without an original source present.
Quite.
For example, if one wants to build binaries with sbuild, it is (right
now, anyway) necessary to build a source package because that's how
sbuild transports the source into the build environment. Right now I,
horrifyingly, have to advise [1] users that in some circumstances they
should run this command:
sbuild -c stretch -A --no-clean-source \
--dpkg-source-opts='-Zgzip -z1 --format=1.0 -sn'
[1] https://manpages.debian.org/testing/dgit/dgit-user.7.en.html#Using_sbuild
Simon Richter writes ("Re: Git Packaging: Native source formats"):
> On Thu, Aug 29, 2019 at 09:42:50PM +0200, Philipp Kern wrote:
> > Obviously I'm not bound to that format being "3.0 (native)" but some
> > "3.0 (dumb)" that just tars up the whole tree without caring about the
> > version scheme would then be nice to have as a replacement. ;-)
>
> Are you planning to upload these?
Obviously not to Debian. I don't think that invalidates the point.
Users are supposed to be able to modify and build software on their
own systems without any expectation that the result will go to Debian.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Theodore Y. Ts'o writes ("Re: Git Packaging: Native source formats"):
> On Thu, Aug 29, 2019 at 11:23:01AM +0100, Ian Jackson wrote:
> > I think dgit ought to be compatible with the idea of shipping
> > upstream's .asc's about, but maybe there are bugs. I don't ever do
> > this so I don't know if it works and I doubt there are tests for it.
> >
> > So, if you have a package where you want to use dgit push and you find
> > the upstream .asc is not being included, please file bug(s).
>
> The problem I have is that "dgit gbp" doesn't extract the upstream
> .asc. Not a big deal, I use /tmp/gbp for my build directory, and I
> manually checkout and populate it with the .asc file. But building
> from "dgit clone" won't generate same package as I do (which includes
> the .asc file for the orig.tar.gz file.)
When you say
building from "dgit clone" won't generate same package as I do
(which includes the .asc file for the orig.tar.gz file.
I'm not an expert on .asc handling but I think what you mean is
Steps to reproduce
0. mkdir bpd
1. dgit --build-products-dir=../bpd clone FOO
2. cd FOO
3. dch -i wombat
4. git commit -a -m changelog
3. dgit --build-products-dir=../bpd gbp-build
Expected behaviour
The generated .changes, and the generated .dsc contain
FOO_VSN.orig.tar.gz.asc, just as they are in uploads not
made with dgit.
Observed behaviour
The .asc is not mentioned in either the generated .dsc or the
generated .changes.
? That would be a bug. Can you file it and tell me the value of
`FOO' please :-) ?
(I'm not sure exactly what you mean by "doesn't extract the upstream
.asc" since I don't know what it would be "extracted" from. I tried
to see if I could reproduce for myself, but a quick check through this
subthread didn't turn up the package name.)
Ian.
PS hardcoding directories in /tmp in one's finger macros usually
amounts to embedding tmpfile race vulnerabilities in one's brain.
Even if one's own machine has per-user /tmp, one's wetware becomes
vulnerable when combined with a differently configured computer. IMO
one should make and use a ~/tmp or something.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Ian Jackson writes ("Re: Git Packaging: Native source formats"):
> Simon McVittie writes ("Re: Git Packaging: Native source formats"):
> > Unlike 1.0 (non-native) vs. 3.0 (quilt), where some people prefer one and
> > some people prefer the other, I am not aware of any advantages of 1.0
> > (native) over 3.0 (native). If 3.0 (native) is indeed strictly better
> > than 1.0 (native), perhaps it would be reasonable to say that packages
> > that intentionally have a non-native version number but a native
> > source format should declare this explicitly, by using "3.0 (native)"
> > in d/source/format? That way, if a version 1.0 source package has a
> > non-native version number, tools can assume that it was meant to have
> > a .orig, and issue warnings; conversely, if a source package with a
> > non-native version number explicitly has "3.0 (native)" format, tools
> > could assume that the maintainer wants what they asked for.
>
> Perhaps. I have a vague feeling that there might be (or have once
> been?) some reason to prefer 1.0 (native) to 3.0 (native) but I can't
> bring it to mind, and now that I try to think about it it's all just
> fog. Maybe I am remembering some years-old abundance of caution.
Private email prompted me to check something, and I was right to
remember that there was a difficulty. dpkg-source refuses to create a
`3.0 (native)' package with a Debian revision number:
$ dpkg-source -b .
dpkg-source: error: can't build with source format '3.0 (native)': native
package version may not have a revision
$
It looks like the dpkg maintainers are adamant that dpkg-source is
correct. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=737634
So `3.0 (native)' is not strictly better than 1.0. dpkg-source
refuses to work in the situation where I am saying (and you seem to be
agreeing) that it shouldn't even print a warning ...
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Milan Kupcevic writes ("Re: Git Packaging: Native source formats"):
> I've also seen developers deleting a git tag and then creating a new
> git tag using exactly the same name/release number pointing to
> different commit.
It is possible to avoid some of these problems by using a git server
which does not permit tag rewriting and/or branch rewinding.
The dgit git server has these properties: once you have done `dgit
push' and the tag is on the dgit git server, it can't be deleted or
changed. Likewise, the suite branches that you push to are
fast-forwarding. Breaching these rules would involve administrator
intervention; as the administrator I would expect to do that only in
very rare situations, where we need to rewrite history eg to remove
legally dangerous objects.
(Because of the archive's policy about new packages, which permits
reuse of version numbers, there is a corresponding exception for
packages which are still entirely in NEW.)
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Simon McVittie writes ("Re: Git Packaging: Native source formats"):
> On Thu, 29 Aug 2019 at 11:56:55 +0100, Ian Jackson wrote:
> > If you already don't care about bit-identical upstream tarballs, then
> > dealing with these tarballs is a reasonably well-solved problem.
> > git-deborig etc. FTW.
>
> I think it's important to distinguish between the two things that you
> might mean by "bit-identical upstream tarballs": you might not care
> whether the orig tarball is bit-identical to upstream's official release
> artifact (assuming they have one), but you still have to care about
> the orig tarball being bit-identical to any orig tarball of the same
> name that might previously have been uploaded to the Debian archive,
> because otherwise the archive will reject your uploads.
Good point. I meant "bit-identical to upstream".
> Does/can git-deborig guarantee that a given git commit will always
> produce the same tarball?
No, it doesn't. But it uses git-archive and in practice that nearly
always gives you the same tarball - even though that's not guaranteed.
I haven't seen any trouble (but I normally use `dgit clone' or `dgit
fetch' and that arranges for any existing origs in the archive to be
in the right place on my laptop, so perhaps that means I avoid trouble
that others see).
> I think that last point is a significant reason to want the nativeness
> of the source package to match the nativeness of the version number: for
> version 1.0 source packages, detecting a non-native version with a native
> source format is the only way to generate any sort of warning about this.
This is a good point. Of course it's only a warning and warnings are
often ignored when you're in a hurry.
> Unlike 1.0 (non-native) vs. 3.0 (quilt), where some people prefer one and
> some people prefer the other, I am not aware of any advantages of 1.0
> (native) over 3.0 (native). If 3.0 (native) is indeed strictly better
> than 1.0 (native), perhaps it would be reasonable to say that packages
> that intentionally have a non-native version number but a native
> source format should declare this explicitly, by using "3.0 (native)"
> in d/source/format? That way, if a version 1.0 source package has a
> non-native version number, tools can assume that it was meant to have
> a .orig, and issue warnings; conversely, if a source package with a
> non-native version number explicitly has "3.0 (native)" format, tools
> could assume that the maintainer wants what they asked for.
Perhaps. I have a vague feeling that there might be (or have once
been?) some reason to prefer 1.0 (native) to 3.0 (native) but I can't
bring it to mind, and now that I try to think about it it's all just
fog. Maybe I am remembering some years-old abundance of caution.
Sean and I faced an aspect of this problem with tag2upload and 1.0
source format. (We didn't want to just say 1.0 wasn't supported.)
The information in git needs to specify whether the output is supposed
to be a native package. Obviously git-debpush shouldn't look for
tarballs in your `..', like dpkg-source does. So we ask the users of
format 1.0 source packages to add an appropriate setting to
debian/source/options.
Anyway, it seems to me that if debian/source/format says `3.0
(native)' then the warning is inappropriate.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: dput problem: Ancient sha256sum? [and 1 more messages]
Mattia Rizzolo writes ("Re: dput problem: Ancient sha256sum?"):
> That said, the fact that your .orig.tar.gz changed is an indicator
> that you are doing something fishy and you should double check your
> workflow. You should not be able to accidentally end up with a
> different .orig.tar.gz.
I agree with this. That is, Thomas, I would advise you to try to
understand what you did differently to generate the two different orig
tarballs.
When you know why they are different you will know which one you want.
If it's the old one, download it from mentors and arrange for your
re-upload to use it. If it's the new one, delete your upload as Peter
Pentchev suggests and then you can upload your new .orig.
I do have to disagree with something Peter writes:
Peter Pentchev writes ("Re: dput problem: Ancient sha256sum?"):
> [...] Since dmagnetic is a package that is not in
> Debian yet, the first version to be uploaded must have a -1 revision;
> rebuild your package as 0.17-1 and try to upload it again. Then
> mentors.d.n will recognize that you are trying to upload a new version
> of the same package and (unlike the actual Debian archive) will allow
> you to replace the original upstream tarball.
As far as I'm aware there is no requirement for the first version of a
package uploaded to Debian to have a -1 revision. There is certainly
no general requirement that first upload of any particular upstream
version has to have a -1 revision, or even that it must have a -1 or
-0.1 revision.
I think the practice of reusing version numbers for different packages
is confusing. It leads to mistakes, and should be deprecated.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Sam Hartman writes ("Git Packaging: Native source formats"):
> Internet is faster and disks are cheaper.
> I assert this is much less of a concern than it used to be.
We have some extremely large packages. Also we have users in places
where internet is slow and/or expensive.
Even for medium-sized packages, it is not desirable to constantly ship
copies of the source code about. I want to be able to upload from a
train wifi, or when roaming abroad on hideously expensive mobile data.
Having said that, I do have some packages which have an upstream where
I don't bother with orig tarballs and just use a native format.
> Using native source formats (1.0 and 3.0 (native)) is attractive for
> some git workflows. It means you can just export the git repository and
> don't need to make sure that you use the same upstream tarball when
> upstream versions are the same.
> You don't need to synthesize a separate upstream tarball.
If you already don't care about bit-identical upstream tarballs, then
dealing with these tarballs is a reasonably well-solved problem.
git-deborig etc. FTW.
The real cost of non-native packages is not the few commands you have
to type for your own packages. It's the cognitive and codebase cost
of knowing how to deal with them, particularly when you're not the
maintainer. Especially, `3.0 (quilt)' is very complex. So we would
gain something if we could get rid of non-native packages *entirely*.
But we can't.
Note that many maintainers use a bare debian ("packaging only")
packaging workflow which doesn't work (at least with current tooling)
without orig tarballs. You'll have seen messages where some of those
people say they think that is the only tolerable approach.
> I'd like to understand the downsides to doing this:
You have been a bit vague about what exactly you are proposing. If
you are just proposing that native packages should be a strong
recommendation then the benefits are IMO too weak.
OTOH if people feel they "shouldn't" use a native source format for a
small package for some reason, I think that would be worth fixing.
One thing I would like to see changed in this area: some of our tools
issue warnings about packages with a non-native version, but a native
source format. I think there is nothing wrong with this and it should
not be discouraged and doesn't merit a warning. (There is an
awkardness here in that you can sometimes unintentionally generate a
native format source package if your origs are missing...)
HTH.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Theodore Y. Ts'o writes ("Re: Git Packaging: Native source formats"):
> Or if we end up moving to dgit for everything, and we don't want to
> use pristine-tar (which I like, but I realize that's not an opinion
> shared by everyone; some people seem to hate it), and upstream uses a
> non-git repo (say, bzr, or hg) and still uses signed tar.gz files, I'd
> argue we need to have a good way to reserve the cryptographic
> signature of upstream's foo.tar.gz and foo.tar.gz.asc in a dgit-only
> world.
I think dgit ought to be compatible with the idea of shipping
upstream's .asc's about, but maybe there are bugs. I don't ever do
this so I don't know if it works and I doubt there are tests for it.
So, if you have a package where you want to use dgit push and you find
the upstream .asc is not being included, please file bug(s).
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Holger Levsen writes ("Re: tag2upload service architecture and risk assessment
- draft v2"):
> On Wed, Aug 28, 2019 at 05:07:00PM +0100, Ian Jackson wrote:
> > In my proposal the source package is reproducible (in the
> > "reproducible builds" sense) from the uploader's signed git tag.
>
> i'm confused. 'reproducible builds' is about creating bit by bit
> identical binaries from a given source.
>
> if you are talking about re-creating bit by bit identical source
> packages, that's fine, but nothing within the scope of reproducible
> builds.
Sorry for the confusion. When I wrote
reproducible (in the "reproducible builds" sense)
I wasn't saying that this is somehow part of, or within the scope of,
the reproducible builds project. I was just clarifying what the word
"reproducible" meant in my sentence: I am using the word
"reproducible" the same way that the reproducible builds project uses
it - ie I am borrowing that definition of reproducible. (That's what
"X (in the Y sense)" means.)
I was indeed clarifying that I do mean bit-by-bit identical. In this
case, bit-by-bit identical dsc (apart from the signature of course),
from (i) git tag (ii) _source.buildinfo containing tools versions etc.
> also, as a side note, we have tried to reproduce bit by bit identical
> source packages, failed and moved on. it didnt seem trival when we
> tried.
I remember some of those discussions. I'm pretty sure it's possible
in my context, although there are as you say some difficulties with it
in the wider reproducible builds context.
I hope that helps.
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Joerg Jaspert writes ("Re: tag2upload service architecture and risk assessment
- draft v2"):
> First off: I, for personal reasons, am a bit detached right now with
> anything Debian (though that should change soon). For that reason, I
> haven't read most of the mail threads, though i skimmed over this
> one a bit.
Hi. Thanks for your attention.
> I currently do not have too deep a thought on how good their
> implementation is. Just one thing I've seen picked at multiple times,
> and in different places: The current implementation appears to move away
> the final integrity check linking an upload to a person away from the
> archive software to some other.
I don't think that's really true, although I can see why you might
think it from reading the thead. I will try to explain.
> Thats a no-go.
>
> Note: I do not say it must be "a dsc" "a git commit" or "a something"
> that is used for this check. That is an implementation detail. But the
> final check/link of an upload with a maintainer(s key) has to be "in"
> the archive. Systems before it can *additionally* do any number of them,
> but the final one is in dak.
In my proposal, as soon as dak supports it, each upload from the
tag2upload service would include a copy of the original uploader's
signed git tag object.
This would allow dak to verify the identity of the original uploader,
and the uploader's intended source package, version, suite, and
distribution; and see that they intended to use the tag2upload
service. All of that is covered by the original uploader's signature
on the git tag object and can be checked very simply without further
data. Based on what's been said in the thread I imagine that dak
would want to check all of these things.
The difficulty is that it is complicated to verify the *contents* of
the source package. (It wasn't clear to me, from your mail, whether
that is something you think is critical for dak to do.)
It would be *possible* for dak to verify the contents, since the .dscs
generated by the tag2upload service are reproducible. But it's not
easy.
Firstly, it depends on additional data: the referenced git objects
have to be obtained. (There is at least a designated place to get
them, so finding them is not a problem.)
Secondly, it is a complicated process, because it is reproducing the
complicated mapping between maintainer git branches (which may be in a
variety of forms - see my git packaging practices survey [1]) and
uploaded .dscs (which must be in a standard form).
So to do that reproduction dak would have to effectively have another
instance of the tag2upload infrastructure within it. Overall I don't
think this would make sense, even though it's possible.
I hope that clarifies things.
Thanks,
Ian.
[1] https://wiki.debian.org/GitPackagingSurvey
Not all of these are supported by tag2upload yet, but the most
common ones are. The biggest one which is missing is the monorepos.
They present additional difficulties because the maintainer tag name
must include the package name, so can't conform to standard gbp or
DEP-14 practice.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Russ Allbery writes ("Re: tag2upload service architecture and risk assessment -
draft v2"):
> For who-uploads, I think you just need a trusted metadata store somewhere,
> and recovering this from the PGP signatures on *.dsc files is not a great
> trusted metadata store (among other things, it's tedious and complicated
> to search).
Also, my proposal includes the original uploader information in
additional .dsc fields. So wouldn't be hard to teach existing
machineries (which look at the .dsc signer) to use the new
information, and there is no actual need for a new database anywhere.
> The cryptographic binding becomes important if we for some reason don't
> trust archive upload records maintained by DAK, and I'm not sure of a use
> case for that.
In my proposal the source package is reproducible (in the
"reproducible builds" sense) from the uploader's signed git tag. That
is admittedly less convenient to verify than the just checking the
.dsc signature.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Sam Hartman writes ("Re: tag2upload service architecture and risk assessment -
draft v2"):
> I'm sure that Ian and Sean had been thinking about this before the
> DPL campaign. But I think in a very real sense, they took that
> discussion and tried to show us what it might look like.
This is much much older than the DPL campaign.
tag2upload is the next stage in the git plan that was developed on the
famous piece of cardboard in Vaumarcus in 2013. That plan's basic
design principle is *bidirectionally* gatewaying between dscs and git.
> That is, they wanted to move our authoritative source format to Git,
> possibly even getting rid of dscs in the medium future.
I want to be 100% clear that although I think dscs are largely
obsolete, I have no intention of trying to get rid of them any time
soon. Certainly not in the medium term. Maybe not in the long term
either.
There are a lot of unresolved issues with getting rid of dscs
completely. See the recent LFS discussion here on -devel for just one
small example. There will be packages and uploads where the .dsc is
primary for a very long time, perhaps forever.
Additionally, the social stress of trying to abolish dscs would be far
too severe. So let's avoid talk of getting rid of dscs.
My goal is to make it possible for git users, and dsc users, to each
have a good user experience and to collaborate with each other.
Part of that is for git users not to have to worry about or think
about dscs. With the appropriate technology, the worrying about dscs
can be done by computers. git users can spend their brainpower on
commits and code and packaging.
So tag2upload is an important next step for improving the user
experience of git users, and thus lowering friction in Debian.
The DPL campaign discussions made it look like the time might be
right. I had lots and lots of positive conversations in Curitiba.
Generally people were super keen and ready with encouragement,
detailed critical review, etc., according to their nature. My current
proposal includes the results of those conversations.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
e it convenient to verifiably and reproducible and reliably publish the same thing to both the maintainer's git view (which the maintainer regards as primary) and the public archives (which the rest of the project, and downstreams, regard as primary). > Your proposal completely changes the notion of what our package > archive is I can see that that is one way to look at it. As I say above, I think that this change is already well under way, in an uncontrolled and fashion. But, yes, my proposal could be seen as formalising it. > while, IMO, pretending to be something else. I certainly don't want to pretend anything that isn't true. Where would be the best place for me to deal with this more explicitly ? Maybe in my intro web page or as a paragraph in the risk assessment document ? > I don't necessarily assume bad faith, but it feels like you are so > convinced of the righteousness of your approach that you are having > trouble taking concerns seriously. I have spoken to a lot of people who all very much want to upload to Debian just by signing and pushing a git tag (and, implicitly, without messing about with tarballs etc.). Being able to do that would take away large amounts of friction from many people's workflows. That's one reason why I think this is important. I have thought about these matters a lot and it is true that I am by now convinced that this cannot be achieved other than with something roughly like my approach. And I don't think it can be achieved without raising the same concerns as we see here. > I think I don't have a lot more to contribute on the topic. I would appreciate it if you would help me further with your frank criticism. Thanks, Ian. -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Bastian Blank writes ("Re: tag2upload service architecture and risk assessment
- draft v2"):
> We don't want to be forced to trust ftp-master. We have reproducible
> builds to verify the content of binary packages. We have the user
> signatures to verify source packages. Of course none of this are
> foolproof or will work all the time.
>
> Right now the team delegated to keep the archive running and safe is not
> willing to drop the ability to verify the contents independently.
These kind of considerations are why in my proposal the .dscs are
reproducible from the uploader-signed git tags.
Tracing the archive contents back to uploader signatures is already
complicated because of the difficulty of understanding what an
appropriate maintainer signing key is for any particular .dsc. With
my proposal it indeed becomes more complicated.
But I have to ask:
Is the uploader signature on the .dsc really the thing we want to
trace the .dsc back to ? Usually nowadays the uploader .dsc signature
is made on the output of some git-buildpackage rune (or automatically,
by that rune).
Typically the human uploader doesn't intend to release some particular
.dsc; that's an output artefact. The uploader intends to release some
git commit.
So the .dsc should be traceable to that git commit. Currently it is
not. With my proposal the .dsc is traceable to the git history,
including the uploader's signed tag.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Sam Hartman writes ("Re: tag2upload service architecture and risk assessment -
draft v2"):
> Ian Jackson writes:
> > The mapping from git tag to .dsc is nontrivial. git tag to
> > .dsc construction (or verification) is complex and offers a
> > large attack surface to the incoming source code. It ought not
> > to be done near a powerful key such as the dak master archive
> > signing key.
>
> It's nontrivial in your design.
It is true that there are other possible approaches.
Key benefits of the present proposal are:
1. The uploader can continue to use their existing best practice git
repository layout, their existing git workflow, and so on. So no
disruption or degradation of maintainer experience.
2. The Debian ftp archive continues to publish a .dsc like those from
current best practice approaches. So no disruption or degradation
to the experience of downstreams of the archive.
3. tag2upload publishes the git history in accordance with current
best practices (ie, maintainer view on salsa and dgit view on dgit
git server). So no disruption or degradation to the experience of
current git users.
4. The uploader does not need to work with, or transfer or keep, any
tarballs (unless this is needed because of upstream practices). [+]
5. The instruction to upload is a plain signed git tag pointing at
the commit to be released. [*]
1-3 are "no degradation compared to existing approaches".
4-5 are "this is an ordinary git operation, not a bodge where we
transport other data embedded in git objects".
That the conversion to a .dsc is nontrivial seems to follow directly
from the above objectives. It arises because existing conversions of
maintainer's git commits to source packages are already complex.
Oh, and of course:
0. It exists.
Ian.
[+] Publishing pristine upstream tarballs would obviously involve
working with tarballs at least some of the time. It is feasible with
my approach but not yet implemented.
[*] Obviously there has to be some extra information in the tag, such
as instructions about which distro and suite to upload to. But by a
"plain" git tag we mean that any such information is short and flat
and straightforward, so it could be generated by hand, or by a variety
of tools, etc.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Scott Kitterman writes ("Re: tag2upload service architecture and risk
assessment - draft v2"):
> I sometimes use who-uploads from devscripts when I want to find out who
> actually did an upload. In theory, it could be re-written to support
> whatever.
As I mention in my other mail, that's the last entry in my table,
about existing things which look at the .dsc signature getting it
wrong. And yes this would need to be fixed.
> I also check that the signature validates when I download a package
> from the archive. I like the fact that this signature connects to a
> developer key in the keyring.
You presumably don't really notice when it doesn't so connect ?
I guess you (like everyone else) have your tools set to carry on
regardless in that case, because otherwise you will run into failures
(and I don't know what a sensible recovery strategy would be) ?
> That said, I'm not the one who suggested losing this would be a
> problem in the previous thread, so I can't say what they were
> thinking. I just don't think the threat assessment is a serious
> response to what people were suggesting. It would be a mistake to
> assume silence is concurrence.
Do you think my risk assessment is "not a serious response" because I
have missed lots of other things too ? Presumably you don't think
this it is wrong to try to understand what the actual fears are that
motivate the comments.
Maybe I communicated poorly the link between the mailing list mails
and my risk assessment. Do you feel I should have dealt with the
mailing list thread by taking each comment and making a little
subthread ? I could go back and do that of course. I'm not sure it
would be very welcome.
(I'm really not happy to have something I spent some care on, and
which I tried to do with integrity, described as "not a serious
response".)
> I may be wrong, but I think Ian's made up his mind what he wants to do, so
> there's not a lot of point in convincing him otherwise.
Well, it's fair to say that I am reasonably convinced that something
roughly of this shape should be deployed. We need the convenience and
integrity benefits of officially using and publishing the git format
which most of us are unofficially using and unofficially consider
primary.
For the reasons which were explored in Vaumarcus and resulted in the
current design, I think something like my proposal is the only
realistic approach. If you need additional evidence that this is the
case, consider that no other approach has got anywhere beyond mailing
list handwaving - and we have wanted to solve this problem for at
least half a decade.
But that's not to say that I'm not willing to make changes. The draft
that was posted here was already changed significantly from a version
which was reviewed privately by an independent member of a
security-related Debian core team. That private review resulted in
significant changes, notably the addition of privsep. (As a result,
that privsep is the substantial part which is not yet implemented.)
The review of my public v1 draft resulted in my proposals for
facilitating double checks by dak, and general improvements to
traceability.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Scott Kitterman writes ("Re: tag2upload service architecture and risk
assessment - draft v2"):
> I haven't gone back and re-read the previous thread, but I did look at the
> risk assessment and I don't find it a serious response to concerns people
> raised.
I'm sorry you feel like that. But thanks for at least reading it.
> As an example, I recall concerns about there not being an uploader
> signature on the source anymore, so we would lose the ability to
> verify from the archive who was responsible for the upload.
I think I must have read those comments as being solely from the point
of the archive. This is perhaps because I think it is very difficult
to do any kind of post-hoc verification of .dsc signatures: to make
sense of whether a signature is appropriate, you need reliable
information about who was a DD or DM (for each package) at what time.
So I focused on the need for the *archive* to verify things.
So you are right that this point is not captured in my risk
assessment. As I said in the mail at the top of this thread:
| I may well have missed something. Please let me
| know if you think there is anything which is not covered.
Note that verifying .dsc signatures is not an end in itself; it is a
means to some end. The question for me then is what end. There were
some useful answers in this thread.
If I were to add this to the risk table, it would probably look like
this:
Risk
It might be no longer possible to for third parties to usefully
verify the .dsc signature, and thus verify the archive operation and
see who was responsible for the upload.
Degree to which accepted in existing arrangements
Verifying the archive operation is already difficult because .dsc
signatures are valid in the context of an uploader's status as DD/DM
at the time: signature verification may involve using old keyrings,
expired keys, etc.
Control measures and mitigations
Information about the uploader's key identity is included in the
.dsc. The uploader's original signed git tag is permanently
archived (by Debian on a designated server).
Analysis; notably, additional risk?
When the uploader used tag2upload, verification of original
uploaders' signatures will involve obtaining and verifying the
signed git tags; this is additional inconvenience for an
already-unreliable and rarely-done activity.
Would that make sense ? Do you disagree with this characterisation or
analysis of the issue you are raising ?
The question of simply knowing who did the upload (eg, you mentioned
who-uploads from devscripts) is much easier and is more or less dealt
with already under this existing entry:
Communications (eg emails and tracking web pages) which currently go
to (or for the attention of) the signing uploader might go to the
wrong place.
(last entry in the table). Maybe I need to broaden the wording here,
maybe adding "or which refer to" ?
> > Data needed to understand where the .dscs came from might later,
> > become unavailable.
>
> and you describe it as a feature. I really don't know how to
> respond.
I don't think that can really properly cover the issue you mention
above, about .dsc verification.
> I expect technical concerns don't weigh heavily when
> you're on a moral crusade.
Maybe you could assume good faith ? For example you could assume that
I had missed something, as I said I might have done, rather than
implying that I am deliberately missing things out and then shouting
at me. :-(.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Ian Jackson writes ("Re: tag2upload service architecture and risk assessment -
draft v2"):
> [stuff]
Argh. A bunch of people helped me refine this but I sent an early
draft by mistake. I guess it's too late to hope people will read only
the better version, but here it is anyway.
If you haven't read the first one yet, please prefer this one.
Sorry,
Ian.
Sam Hartman writes ("Re: tag2upload service architecture and risk assessment -
draft v2"):
> I do think it would be valuable to confirm whether we're at an impasse.
> It sounds like Ian may think that resolving your concerns would be a
> no-go
I'm definitely trying to have a constructive discussion about the best
design, what the risks are of various approaches, etc.
Indeed I have been trying to resolve people's concerns. The concerns
were almost all about archive integrity, so I have tried to analyse
the actual risks, and where appropriate propose control measures for
those risks.
My intent was not to ignore Bastian's requirements but instead tease
out the risks that they were aimed at, and make constructive
evaluations of those. It seemed reasonably clear what risks Bastian
was concerned with, but I had hoped Bastian would help me out where I
had misunderstood..
Unfortunately it seems that rather than a set of concerns, as you put
it, Bastian has some hard design demands. I didn't mean to cause
offence; I was trying to focus on the exact nature of the problems
these requirements are intended to solve.
>From my reading of the thread, it seems that there are two disputed
design demands, which are related.
The most basic demand is that the archive should be able to verify the
whole contents of the .dsc, given data signed by the user.
The risk assessment explains why I don't think this is an appropriate
requirement, but I will go through it again:
The mapping from git tag to .dsc is nontrivial. git tag to .dsc
construction (or verification) is complex and offers a large attack
surface to the incoming source code. It ought not to be done near a
powerful key such as the dak master archive signing key.
Furthermore, there is nearly no benefit in redoing this mapping. In
my design proposal [1], the conversion occurs on a DSA-managed machine
using fully controlled software, with a copious audit trail.
This contrasts starkly with currently leading approaches for git-based
packaging: currently .dscs are produced from git data by uncontrolled
git-buildpackage runes run on uploader's own systems (not even in a
clean environment, usually). So my proposal is far superior to the
status quo.
In principle it would be possible to satisfy this demand. The
tag2upload service could ship the git data to the archive, bundled
with the .changes (as a git bundle perhaps). The archive would then
re-run the source package generation (perhaps using the reproduction
script that my proposal already includes), and compare the results.
So this is negotiable, although this seems to me like a silly
direction to be going. And it would likely involve a long delay to
deployment if the extra dak rebuild machinery were to be regarded as a
blocker.
The second demand (or the second aspect) is that all of this
verification, by the archive, should be doable without relying on the
git SHA-1 object system.
Again, I have analysed the SHA-1 risk in my risk assessment. So here
too I have explained why I don't think this is an appropriate
requirement to place on the tag2upload service. We have already
accepted the SHA-1 risk; the mitigations we have (including the code
in git which deals with at least the known collision attack) are
tolerably sufficient.
The tag2upload proposal doesn't make it worse; in some ways it is
slightly better because git object data comes from a central source
(salsa) rather than the maintainer's machine.
I think this demand cannot be met by anything I would call a "tag to
upload" system. The tag data would have to contain some kind of file
manifest. The tag would not be constructable by normal tooling, but
only by a special program. It would not be simply a git tag.
> and that you think that his design is a no-go.
> Confirming whether that's true would actually be valuable.
> I think the ball is probably in Ian's court on that issue.
I was hoping for constructive engagement with the substance of the
arguments I am making. I find it difficult to see where to go from
Bastian's most recent message, in which he seemed to me to say he had
been laying down non-negotiable demands and was offended that I
disagreed.
> I also think it would be very interesting to get Joerg's personal
> opinion on designs in this space because it sounds like he's thought
> about it for a while.
I would definitely welcome wider engagement with the substance of the
risks, the design tradeoffs, etc.
> One of the factors we as a project will consider is whether other
> implementations
Re: tag2upload service architecture and risk assessment - draft v2
uch of the heavy lifting in tag2upload is not done by src:dgit itself. In particular, much of the git patch queue manipulation is done using tools from src:git-buildpackage. Thanks, Ian. -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Consensus Call: Git Packaging Round 1
Bernd Zeimetz : > On 2019-08-26 23:41, Sam Hartman wrote: > > I don't think you're part of our consensus. > > Yes, that might be very true. But what you describe by > "our consensus" I'm not a fan of this phrasing by Sam. But he makes a very good point: you have not answered any of the substantive reasons why people might have problems with Gitlab MRs. Taken together with the rest of your mail, it seems to me a bit like you're complaining that people aren't listening (to you, or to some silent majority) but you yourself don't seem to have listened very well to the rest of the thread ? > is the opinion of a few people who actually read this mailinglist > regularily and consider to reply. I really hope that you do not > consider this "consensus" as the opinion of the Debian project. > It is the opinion of 10-20 random people, nothing more. I'm not sure how else you would measure the rough consensus. You'd prefer a GR ? > The proper way to handle this would be to amend > https://dep-team.pages.debian.net/deps/dep14/ > and drive it forward. The DEP process provides a framework for documenting proposed changes, etc., but I'm not sure how a DEP proponent would judge consensus other than with traditional methods such as a consensus call here on -devel. > And therefore I see absolutely no reason why the developer > reference should be changed based on this. I'm confused. In your earlier message you were proposing that "3. If a package is maintained on salsa, maintainers have to process merge requests". That would be a change to the DR, presumably. Now you are saying that no change should be made to the DR because this mailing list is not a good place to judge rough consensus ? I think Sam's proposed change would be to document in the DR that a maintainer should handle change requests (including code contributions) sent to the BTS. That would surely just be documenting our existing norm. It seems to me that if you want to change a norm, it is up to you to argue for it. > Please remember that it should be easier and more fun to contribute > to Debian. Keeping packaging in the stone age jsut because some > people still live there is not what you should strive for. Please avoid pejorative language like "stone age". And please avoid describing workflows as obsolete when there are people making cogent arguments why those established workflows are (at least in some respects) better than newer ones. Ian. -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: tag2upload service architecture and risk assessment - draft v2
Ian Jackson writes ("tag2upload service architecture and risk assessment -
draft v2"):
> Thanks for all the comments on the draft service architecture I posted
> in late July. [1] I have made a v2, incorporating the various helpful
> suggestions, and the information from the thread.
It has been a week. I'm not sure whether that means people haven't
looked at the linked documents (I guess there might be some tl;dr
going on).
It would be good to get a review of the risk analysis in particular,
hence this ping. I'd appreciate public follow-ups.
Anyway, I'll leave it at least another week. If anyone wants more
time than that to properly review my proposal, please let me know when
you expect to be able to respond.
If no-one has any comments, then I intend to use this set of documents
as basis for asking for a formal go-ahead before I do a lot of
implementation work (privsep code etc.)
Thanks,
Ian.
> Some respondents raised archive integrity concerns. It seemed best to
> address those more formally, and in a more structured way, than as a
> mailing list subthreads. Accordingly, v2 of my proposal has a formal
> risk assessment, in a format loosely borrowed from health and safety
> management.
>
> I think I have captured in the risk assessment all the risks mentioned
> in the thread, but I may well have missed something. Please let me
> know if you think there is anything which is not covered. Also please
> let me know if any of my analysis seems wrong.
>
> Please find an introduction, and detailed documentation, here:
> https://people.debian.org/~iwj/tag2upload/2019-08-20/
>
> Thanks,
> Ian.
>
> [1] This message and the subsequent thread:
> https://lists.debian.org/debian-devel/2019/07/msg00501.html
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Consensus Call: Git Packaging Round 1 [and 1 more messages]
Andrej Shadura writes ("Re: Consensus Call: Git Packaging Round 1"):
> I noticed some people [citation needed] think it is not important to
> preserve pristine upstream tarballs with the move to Git, and it's
> okay to regenerate them from a Git branch without trying to preserve
> checksums of the tarballs upstream has somehow generated.
I am one of these people. I have always been sceptical of the need to
preserve upstream pristine tarballs. I haven't been vocal about this
because no-one is forcing anyone to publish pristine tarballs. So in
any situation where the maintainer doesn't want to pay the costs of
preserving pristine upstream tarballs, the maintainer can simply not
do so.
That overall stance has a lot of social value for the project, because
it means we can all cooperate without having to have this debate. We
can save our energy for doing something more useful.
For myself I try to publish pristine upstream tarballs when it's
reasonably convenient, because I feel that this is a thing that some
people value, even though I myself think the value is very limited.
That's part of playing nice in a community like Debian.
If we are going to mandate something - or even, if we are going to
change our current stance (which seems to be that this is a "nice to
have"), then a discussion of the upsides and downsides - particularly,
with a practical focus - is necessary.
> I just had an impression this is not being considered and sort of
> assumed a consensus that we won't keep them in Git when we move to
> git-debpush workflows. Let's discuss it instead of have people
> assuming things potentially incorrect :)
Adoption of git-debpush will be optional, obviously. I think reasons
to actively encourage it over `dgit push-source' [1] are rather
limited, at least right now.
Also, in principle git-debpush and the tag2upload service could
support eg pristine-tar, although that would involve quite some
development work. If someone wants to lead that, I would be happy to
review patches, do supporting work, etc.
So certainly, speaking for myself, I don't intend to discourage or
impede anyone from publishing pristine tarballs, even though as I say
I think their value is very limited.
Sam Hartman writes ("Re: Consensus Call: Git Packaging Round 1"):
> That's on my list for discussion in round 3.
I have carefully avoided any discussion of the merits in my message,
and am happy to wait.
Ian.
[1] Both `dgit push-source' and `git debpush' publish your actual git
history on the dgit git server in a form useful for users, so I have
argued, and will continue to argue, that there are good reasons to
encourage maintainers to do one of these things.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Accepted vacation 3.3.3 (source) into unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Format: 1.8 Date: Sat, 24 Aug 2019 11:47:49 +0100 Source: vacation Binary: vacation Architecture: source Version: 3.3.3 Distribution: unstable Urgency: medium Maintainer: Phil Brooke Changed-By: Ian Jackson Description: vacation - email autoresponder Changes: vacation (3.3.3) unstable; urgency=medium . * Re-upload as a source-only upload to allow testing migration. Checksums-Sha1: 910f2e6ba0025c4eb8a1460a61f85684659df3ef 1224 vacation_3.3.3.dsc a7fad55f926007201cdccb91f6cb570d264bc7ef 12665 vacation_3.3.3.tar.gz Checksums-Sha256: 11cc1e49eecd29ca78b9737e4534d0623ecd3b1186fd95e46f72cf549979aeb3 1224 vacation_3.3.3.dsc 0bd52d8d3df77f2079c7a42cf530c76d8ed3f135d7f46a8be74829a321d08f04 12665 vacation_3.3.3.tar.gz Files: 72c9015600999d1720439d0e4abcc4b4 1224 mail optional vacation_3.3.3.dsc d8a851f041073417b33e632fb14df3c7 12665 mail optional vacation_3.3.3.tar.gz -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAl1hFtMACgkQ4+M5I0i1 DTkt6ggAkXoJaR2NGTVkvDDf77qhd7dHzszs9H3PHDbZkl+uh3cY1TSUXMmGBafz IMa/RlDXra6AMsgWgxTX2Dqkw3db3SMEokUVFSVcy+jxLvnWxLPigAn0S9LN00m/ 9fwx++g32624weNZRqSH+H25YAAiJ8x5k29QUmFtguOBAwLyxPFv/4uW8RDAPA+0 54ZLKsiZcXxC2yS88pKrRgYMU0jp4ORR8zFWMCRnG2Od0EUSkQb4cjDZKxi7kna3 CzcNapWvwEC052re3yj02Z4wPzaxoGkfehyVOWkP5Unl4AKYdQCxjayS5dj+4owa ZEk/OtJYk0skRJ+0Bnz/08r8I0trEg== =4eDw -END PGP SIGNATURE-
tag2upload service architecture and risk assessment - draft v2
Thanks for all the comments on the draft service architecture I posted in late July. [1] I have made a v2, incorporating the various helpful suggestions, and the information from the thread. Some respondents raised archive integrity concerns. It seemed best to address those more formally, and in a more structured way, than as a mailing list subthreads. Accordingly, v2 of my proposal has a formal risk assessment, in a format loosely borrowed from health and safety management. I think I have captured in the risk assessment all the risks mentioned in the thread, but I may well have missed something. Please let me know if you think there is anything which is not covered. Also please let me know if any of my analysis seems wrong. Please find an introduction, and detailed documentation, here: https://people.debian.org/~iwj/tag2upload/2019-08-20/ Thanks, Ian. [1] This message and the subsequent thread: https://lists.debian.org/debian-devel/2019/07/msg00501.html -- Ian JacksonThese opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Re: Bits from the Release Team: ride like the wind, Bullseye!
Lisandro Damián Nicanor Pérez Meyer writes ("Re: Bits from the Release Team:
ride like the wind, Bullseye!"):
> My personal point of view (and because of this it might be biased)
> is that the maintainers of the packages that ship autopkgtest should
> be the reponsibles for any breackage it might occur on them because:
>
> - They added autopkgtests, so they are showing an intent on
> reviewing them when they fail.
> - They will certainly know their packages better than the library
> maintainer, and thus they have more chances to get the root of the
> issue sooner. Of course that might mean finding a bug in the
> library, but that's just ok.
In the general case the proper investigation of a bug might need
involvement from both people, collaboratively. That involves a kind
of ping pong on a technical level.
> On 19/08/08 09:46, Paul Gevers wrote:
> > I think we should also try to improve the visibility towards reverse
> > dependencies that their autopkgtest is blocking other packages. I would
> > love tracker (and the old pts) to show this on their page. (Working on
> > such a patch is on my TODO list, except not at the top).
I already made grep-excuses print this information. It has been very
helpful to me. Maybe we should make --autopkgtests the default ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Building GTK programs without installing systemd-sysv?
Adam Borowski writes ("Re: Building GTK programs without installing
systemd-sysv?"):
> On Wed, Aug 14, 2019 at 01:47:47PM +0200, Philipp Kern wrote:
> > You want dbus-x11 instead of dbus-user-session then, I think.
>
> For this and related issues, I wonder if it would be better to replace the
> virtual package pair of {default-,}dbus-session-bus with a real dummy
> package that has:
> Depends: dbus-x11 | systemd-sysv, dbus-user-session | dbus-x11
>
> This would pull in dbus-user-session for systemd users, dbus-x11 for any
> other init.
>
> (The idea might be bogus -- I haven't slept since Monday morning; anyone up
> for beersigning near Frankfurt airport until late evening?)
I didn't see any response to this and I don't immediately see why it
would be a bad idea.
Obviously it's slightly a bodge but these kind of issues involving
dbus-user-session seem common enough that it would probably be worth
doing this as a workaround, if it would be effective. (Which it seems
like it probably would be.)
Adam, have you tested this (on both systemd and sysvinit systems) ?
Thanks,
Ian.
(at the Worldcon in Dublin so not got the bandwidth to check this
myself.)
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: salsa.debian.org partially down
gregor herrmann writes ("Re: salsa.debian.org partially down"):
> Ack. And that's all I wanted to say: that I found Ian's term "foolish
> user action" inappropriate in this case of an accidental DOS.
Err, yes. I retract that comment. I had misunderstood what had
occurred.
Sorry,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
