Re: Core-updates merge

[Development of GNU Guix and the GNU System distribution.]

Hi Pierre,

On Fri, Jul 14, 2023 at 11:18 AM Pierre Langlois
 wrote:
>
> Attribution is always nice of course, but mistakes happen

Thank you for your kind and generous reaction to the innocent oversight.

Please have a good weekend!

Kind regards
Felix

Re: Core-updates merge

[John Kehayias]

Hi Pierre!

On Fri, Jul 14, 2023 at 07:12 PM, Pierre Langlois wrote:

> Hi John!
>
> John Kehayias  writes:
>
>> Bringing back up an old thread, but
>>
>> On Fri, Apr 28, 2023 at 05:55 AM, John Kehayias wrote:
>>
>>> Dear Andreas and fellow Guix-ers,
>>>
>>> On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:
>>>
>> [...]
 Each and every package is not yet in shape; please feel free to submit
 patches for your favourite packages that fail to build. In particular:
 - python-yubikey-manager does not build currently; work to correct this
   is underway.
>>>
>>> I've just submitted  which does a bit
>>> more than just fix that package and would be good for a Python feature
>>> branch. I'll send the cover letter to this list for wider visibility,
>>> though the Python team was cc'ed on the series. I suppose I should add
>>> myself to the team after this.
>>>
>>
>> Thanks to Pierre Langlois python-yubikey should now finally be fixed
>> on master, without all the big python build changes. (Poetry is still
>> broken.) This was via . Unfortunately
>> I managed to clobber the author line in the git log after editing and
>> testing locally, sorry about that! (Anything that can be done to fix
>> that?)
>
> Thank you for picking up the series!  I admit I had forgotten I had sent
> it, I don't have too much time for Guix these days I'm afraid.
> Attribution is always nice of course, but mistakes happen, especially
> with git, so I wouln't worry about it in this instance! At least
> personnally I don't mind :-), just happy that the patches went in.
>

Thanks for the patches! I was so caught up in all the other python
stuff I ran into at the time that I never looked to pull out the fixes
for python-yubikey-manager (which I use and had been using from my
local branch).

I appreciate the understanding! We do have your copyright lines in
these additions. Given the overlap and complementary nature of both
patch series (I hadn't done your cleanup but did other things) I
should have just had us both as co-authors clearly from the start.

I'll give myself a slight pass since I think this was the first
non-trivial thing I pushed for someone else (working on a separate
branch locally, rebasing several times to make fixes, etc.). Hopefully
that means the next ones will be smoother and I can get rolling on
helping out with reviews and pushing our many great patches we have
waiting.

Thanks again for the work and understanding and Guix is always happy
to have your contributions!

John

Re: Core-updates merge

[John Kehayias]

Hi Felix,

On Fri, Jul 14, 2023 at 09:52 AM, Felix Lechner wrote:

> Hi John,
>
> On Fri, Jul 14, 2023 at 9:37 AM John Kehayias
>  wrote:
>>
>> Unfortunately
>> I managed to clobber the author line in the git log after editing and
>> testing locally, sorry about that! (Anything that can be done to fix
>> that?)
>
> Giving proper attribution can be essential in group projects. You
> could revert your commit and then re-commit the change with the
> correct author.
>

I agree, which is why I had a minor panic upon seeing the wrong name
in the git log on savannah after pushing (after having checked locally
so much to make sure I got it right, I must have glossed over it in
the final formation).

Revert/re-commit seems how we have to handle it, thanks for that note.

> How to handle it best depends on your dynamics with the injured party.
> I personally never cared enough—especially not in the case of an
> innocent mistake—but I know that some folks feel strongly about being
> made whole.
>

I made sure Pierre was aware in the original patch thread and here,
and see a response now...

> Kind regards
> Felix

Thanks Felix!
John

Re: Core-updates merge

[Pierre Langlois]

Hi John!

John Kehayias  writes:

> Bringing back up an old thread, but
>
> On Fri, Apr 28, 2023 at 05:55 AM, John Kehayias wrote:
>
>> Dear Andreas and fellow Guix-ers,
>>
>> On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:
>>
> [...]
>>> Each and every package is not yet in shape; please feel free to submit
>>> patches for your favourite packages that fail to build. In particular:
>>> - python-yubikey-manager does not build currently; work to correct this
>>>   is underway.
>>
>> I've just submitted  which does a bit
>> more than just fix that package and would be good for a Python feature
>> branch. I'll send the cover letter to this list for wider visibility,
>> though the Python team was cc'ed on the series. I suppose I should add
>> myself to the team after this.
>>
>
> Thanks to Pierre Langlois python-yubikey should now finally be fixed
> on master, without all the big python build changes. (Poetry is still
> broken.) This was via . Unfortunately
> I managed to clobber the author line in the git log after editing and
> testing locally, sorry about that! (Anything that can be done to fix
> that?)

Thank you for picking up the series!  I admit I had forgotten I had sent
it, I don't have too much time for Guix these days I'm afraid.
Attribution is always nice of course, but mistakes happen, especially
with git, so I wouln't worry about it in this instance! At least
personnally I don't mind :-), just happy that the patches went in.

Thanks,
Pierre


signature.asc
Description: PGP signature

Re: Core-updates merge

[Development of GNU Guix and the GNU System distribution.]

Hi John,

On Fri, Jul 14, 2023 at 9:37 AM John Kehayias
 wrote:
>
> Unfortunately
> I managed to clobber the author line in the git log after editing and
> testing locally, sorry about that! (Anything that can be done to fix
> that?)

Giving proper attribution can be essential in group projects. You
could revert your commit and then re-commit the change with the
correct author.

How to handle it best depends on your dynamics with the injured party.
I personally never cared enough—especially not in the case of an
innocent mistake—but I know that some folks feel strongly about being
made whole.

Kind regards
Felix

Re: Core-updates merge

[John Kehayias]

Bringing back up an old thread, but

On Fri, Apr 28, 2023 at 05:55 AM, John Kehayias wrote:

> Dear Andreas and fellow Guix-ers,
>
> On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:
>
[...]
>> Each and every package is not yet in shape; please feel free to submit
>> patches for your favourite packages that fail to build. In particular:
>> - python-yubikey-manager does not build currently; work to correct this
>>   is underway.
>
> I've just submitted  which does a bit
> more than just fix that package and would be good for a Python feature
> branch. I'll send the cover letter to this list for wider visibility,
> though the Python team was cc'ed on the series. I suppose I should add
> myself to the team after this.
>

Thanks to Pierre Langlois python-yubikey should now finally be fixed
on master, without all the big python build changes. (Poetry is still
broken.) This was via . Unfortunately
I managed to clobber the author line in the git log after editing and
testing locally, sorry about that! (Anything that can be done to fix
that?)

Now to return to the big python series, minus a few patches.

John

Re: Core-updates merge

[Ludovic Courtès]

Hello!

"Leo Famulari"  skribis:

> Thank you for leading this effort!
>
> The value of leadership and project management is sometimes underappreciated 
> in software projects, but they are essential for our success.

Agreed!  You did a great job Andreas at coordinating all the work *and*
actually doing a lot of it—thank you, and thanks to everyone who
tirelessly worked on fixing things!  Really happy to be enjoying the
fruits of this hard labor now.  :-)

Ludo’.

ocaml bootstrap (was Re: Core-updates merge)

[Efraim Flashner]

On Fri, Apr 28, 2023 at 04:17:40PM +0200, Simon Tournier wrote:
> Hi Andreas,
> 
> On mar., 25 avril 2023 at 16:09, Andreas Enge  wrote:
> 
> > - OCaml could be simplified by dropping version 4.07 (Julien Lepiller).
> 
> Well, 4.07 is the version that is de-bootstrapped, i.e. bootstrapped
> using ’camlboot’ via Guile – for details see [2].
> 
> However, higher versions (4.09, 4.14, 5) does not use this seed and thus
> they are not de-bootstrapped.  Well, I do not know the status upstream;
> from my point of view, we have two options:
> 
>  a) Agree with other distros and OCaml folks to rely on a common OCaml
>  4.07 bootstrapped using camlboot and then use this OCaml 4.07 as the
>  seed for the subsequent versions.  Somehow having a way to verify the
>  current OCaml compiler without running again and again via camlboot.
> 
>  b) Build ourselves a chain from 4.07 bootstrapped with camlboot to
>  modern OCaml compilers.  However, each time we modify one dependency of
>  camlboot, it means rebuild the complete chain.  Well, bootstrapping via
>  camlboot can be very slow and I do not know if we have the resources
>  for non-x86_64 architecture.  Here, the list of the emerged
>  dependencies:

I think similarly to the very slow *-mes part of the bootstrap chain, we
have to realize that some parts are just very slow. (A vote for b)

I built camlboot once on aarch64, probably a year ago. IIRC it took more
than 24 hours. I will say that it is doable without needing 8GB of ram,
which is also a limiting factor for the slow architectures.

> $ guix graph camlboot -t bag-emerged | grep label | cut -d'=' -f2
>  "camlboot@0.0.0-1.45045d0", shape 
>  "guile@3.0.9", shape 
>  "pkg-config@0.29.2", shape 
>  "tar@1.34", shape 
>  "gzip@1.12", shape 
>  "bzip2@1.0.8", shape 
>  "file@5.44", shape 
>  "diffutils@3.8", shape 
>  "patch@2.7.6", shape 
>  "findutils@4.9.0", shape 
>  "gawk@5.2.1", shape 
>  "sed@4.8", shape 
>  "grep@3.8", shape 
>  "xz@5.2.8", shape 
>  "coreutils@9.1", shape 
>  "make@4.3", shape 
>  "bash-minimal@5.1.16", shape 
>  "ld-wrapper@0", shape 
>  "binutils@2.38", shape 
>  "gcc@11.3.0", shape 
>  "glibc@2.35", shape 
>  "glibc-utf8-locales@2.35", shape 
>  "libffi@3.4.4", shape 
>  "bash-minimal@5.1.16", shape 
>  "libunistring@1.0", shape 
>  "libgc@8.2.2", shape
>
>  c) Fix the dependencies of camlboot.

Looking at camlboot, there's a native input of guile-3.0. Unless we want
to bootstrap using some of the boot0 packages or actually find the
minimum set of packages needed and remove the rest that are brought in
by implicit-inputs (recursively) I think it's already as small as
possible.

> 
> Well, it seems a separated discussion but it echoes the recent blog post [3]
> about “The Full-Source Bootstrap”. :-)
> 
> 2: 
> https://10years.guix.gnu.org/video/camlboot-debootstrapping-the-ocaml-compiler
> 3: 
> https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/
> 
> 
> Cheers,
> simon
> 
> 
> 

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Build dependency inflation (was: Re: Core-updates merge)

[Christopher Baines]

Simon Josefsson via "Development of GNU Guix and the GNU System distribution." 
 writes:

> [[PGP Signed Part:Undecided]]
> Andreas Enge  writes:
>
>> - Too much in Guix depends on too much else, which makes building things
>>   needlessly entangled; in particular time zone data should not be referred
>>   to by packages, but be loaded at runtime (Leo Famulari).
>
> This is an important open problem -- is there any way to attack this
> problem in a systematic way?  I guess it is hard to understand which
> packages ends up depending on what since it is a large graph with long
> cycles, and also to understand which build depencies are essential and
> which are superficial, and thus consequently challenging to know where
> to start working to reduce build dependencies?
>
> I wonder if it is possible to graph all the build dependencies, and do
> something like a monte-carlo what-if simulation: randomly pick one
> build-dependency from the entire build-dependency graph and remove it,
> and recompute the graph.  If the difference between these two graphs
> leads to a significantly lower total build computational cost than
> before, we may be on to something.  My theory is that "true" build
> dependencies will show up in so many places that removing just one
> instance of it will not affect total build time.  But "needless" build
> dependencies may occur just once or few times, and this approach would
> catch low-hanging fruit to work on.  Maybe the simulation needs to be
> done on more than just removing one build-dependency, it could play
> what-if games removing two build-dependencies etc, or three random
> build-dependencies, and so on.  Maybe my idea is flawed, and this will
> only lead to a list of build-dependencies that are impossible to get rid
> off anyway.
>
> Is there some other method to understand what build dependencies would
> be important to remove, to speed up total rebuild time?
>
> Maybe we could analyze how much of a particular build-dependency
> actually is used by a particular build?  By looking into file-access
> patterns etc.

This is something I'd like to see incorporated in to qa.guix.gnu.org for
patch (and branch) review. While one off analysis is good, I think it's
most important to be able to look at changes and see how they change the
situation.


signature.asc
Description: PGP signature

Re: Build dependency inflation (was: Re: Core-updates merge)

[Simon Tournier]

Hi,

On jeu., 27 avril 2023 at 19:56, Simon Josefsson via "Development of GNU Guix 
and the GNU System distribution."  wrote:
> Andreas Enge  writes:
>
>> - Too much in Guix depends on too much else, which makes building things
>>   needlessly entangled; in particular time zone data should not be referred
>>   to by packages, but be loaded at runtime (Leo Famulari).
>
> This is an important open problem -- is there any way to attack this
> problem in a systematic way?  I guess it is hard to understand which
> packages ends up depending on what since it is a large graph with long
> cycles, and also to understand which build depencies are essential and
> which are superficial, and thus consequently challenging to know where
> to start working to reduce build dependencies?

Hum, I am not to get the “large graph with long cycles” part.  Since the
graph for packages is a DAG.

Well, “guix graph” and “guix graph --path” help to spot the chain of
dependencies between two packages that seem unrelated.

>From my point of view, we should increase the tooling to inspect these
graphs.  Other said, add feature to “guix graph”.

At some point, it could be nice to have “guile-igraph” a Guile binding
of igraph [1].

Last, back on December [2], I started to apply some well-known network
algorithm (link analysis, centrality measure, pagerank, etc.) to the
graph of packages.  From my point of view, these kind of tools could be
very helpful to spot out the packages that need some specific care.

Maybe it would fit a project for a student. ;-)


1: https://igraph.org/
2: https://yhetil.org/guix/874ju4qyd4@gmail.com
   Some stats about the graph of dependencies
   Fri, 09 Dec 2022 18:29:43 +0100
   id:874ju4qyd4@gmail.com


Cheers,
simon

Re: Core-updates merge

[Simon Tournier]

Hi Andreas,

On mar., 25 avril 2023 at 16:09, Andreas Enge  wrote:


> I have just merged core-updates into master and deleted the branch!

Awesome!  Thank you for your patient leadership over the past months. :-)


> - R on powerpc64le needs to be built by changes to valgrind and lz4
>   (Simon Tournier, I).

Now, we are good, right?


> - There is still work to do to bootstrap GHC until the latest version on
>   i686, and to potentially shorten the bootstrap chain (Lars-Dominik Braun).

Argh, I will try to finish my patches soon.  I have:

 + turned off the test suite for ’ghc’ – it allows to not be blocked,
 + introduced ’ghc-testsuite’ to run the test suite,
 + introduced ’ghc-toolchain’ which depends on ’ghc’ and ’ghc-testsuite’
   – well, ’ghc’ being hidden.  Something similar as GCC.
 + shorten the chain as discussed elsewhere [1].

1: https://yhetil.org/guix/87r0siem5c@gmail.com


> - OCaml could be simplified by dropping version 4.07 (Julien Lepiller).

Well, 4.07 is the version that is de-bootstrapped, i.e. bootstrapped
using ’camlboot’ via Guile – for details see [2].

However, higher versions (4.09, 4.14, 5) does not use this seed and thus
they are not de-bootstrapped.  Well, I do not know the status upstream;
from my point of view, we have two options:

 a) Agree with other distros and OCaml folks to rely on a common OCaml
 4.07 bootstrapped using camlboot and then use this OCaml 4.07 as the
 seed for the subsequent versions.  Somehow having a way to verify the
 current OCaml compiler without running again and again via camlboot.

 b) Build ourselves a chain from 4.07 bootstrapped with camlboot to
 modern OCaml compilers.  However, each time we modify one dependency of
 camlboot, it means rebuild the complete chain.  Well, bootstrapping via
 camlboot can be very slow and I do not know if we have the resources
 for non-x86_64 architecture.  Here, the list of the emerged
 dependencies:

$ guix graph camlboot -t bag-emerged | grep label | cut -d'=' -f2
 "camlboot@0.0.0-1.45045d0", shape 
 "guile@3.0.9", shape 
 "pkg-config@0.29.2", shape 
 "tar@1.34", shape 
 "gzip@1.12", shape 
 "bzip2@1.0.8", shape 
 "file@5.44", shape 
 "diffutils@3.8", shape 
 "patch@2.7.6", shape 
 "findutils@4.9.0", shape 
 "gawk@5.2.1", shape 
 "sed@4.8", shape 
 "grep@3.8", shape 
 "xz@5.2.8", shape 
 "coreutils@9.1", shape 
 "make@4.3", shape 
 "bash-minimal@5.1.16", shape 
 "ld-wrapper@0", shape 
 "binutils@2.38", shape 
 "gcc@11.3.0", shape 
 "glibc@2.35", shape 
 "glibc-utf8-locales@2.35", shape 
 "libffi@3.4.4", shape 
 "bash-minimal@5.1.16", shape 
 "libunistring@1.0", shape 
 "libgc@8.2.2", shape

 c) Fix the dependencies of camlboot.


Well, it seems a separated discussion but it echoes the recent blog post [3]
about “The Full-Source Bootstrap”. :-)

2: 
https://10years.guix.gnu.org/video/camlboot-debootstrapping-the-ocaml-compiler
3: 
https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/


Cheers,
simon

Re: [Rust Team] Status post core-updates merge

[Maxim Cournoyer]

Hi Efraim,

Efraim Flashner  writes:

[...]

>> I'm also curious, since you're "early adopters": how are you managing
>> with the team workflow?  Any tips, remarks or impressions that you could
>> share with other teams?
>
> Speak in the plural and no one will realize it's a team of one ;)
>
> I'm thinking of tagging the rust stuff with [rust team] so it stands out
> and is easily searchable. if/when I send patches for other teams I'll
> probably tag those too and hope it catches on.

Sounds like something to automate using git, the way CC'ing team members
can already be automated (see the patches in #58813).

Thanks for working on improving our (far from optimal) Rust situation.

-- 
Thanks,
Maxim

Re: Core-updates merge

[John Kehayias]

Dear Andreas and fellow Guix-ers,

On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:

> Hello all,
>
> I have just merged core-updates into master and deleted the branch!
> This has been a long adventure, which became particularly intensive
> after the last Guix Days in February. First and foremost many thanks to
> everyone who contributed to the branch, be it by commits, discussions or
> by working on the infrastructure.
>

As others have said, a big thank you to you once again for leading the
charge! Much appreciated!

> Each and every package is not yet in shape; please feel free to submit
> patches for your favourite packages that fail to build. In particular:
> - python-yubikey-manager does not build currently; work to correct this
>   is underway.

I've just submitted  which does a bit
more than just fix that package and would be good for a Python feature
branch. I'll send the cover letter to this list for wider visibility,
though the Python team was cc'ed on the series. I suppose I should add
myself to the team after this.

> - R on powerpc does not build; this will also be corrected soon;
> - aarch64 has very few substitutes; I think this is mainly due to the build
>   farm catching up and not so much to packages not building, but this is
>   difficult to know.
> If any of these are essential to you, you may wish to wait a little bit
> longer before a "guix pull", or for the time being pull to a commit just
> before the merge by issuing
>guix pull --commit=472706ae2f9160833951a4e4bcc4c206e03097b0
>
> Every end of a story is the beginning of many new ones. Several areas of
> work have already been identified; I will summarise what came up on the
> mailing list and who expressed interest to work on it - please feel free
> to join if a topic interests you, or launch another initiative if you
> want to work on a different subject!
> - rust-team already has a branch that is almost ready to be built and
>   merged, as a precursor to the team based workflow that we need to invent
>   (Efraim Flashner).
> - R on powerpc64le needs to be built by changes to valgrind and lz4
>   (Simon Tournier, I).
> - Many Python packages need updates, in particular with the aim of building
>   python-yubikey-manager (John Kehayias, Lars-Dominik Braun, Brian Cully).

I'll have to see what other patches we want to add to my series for
the branch and creating/building that as soon as possible. There is
some feedback needed on the deeper changes in my patch series, but
hopefully won't take much to finalize.

> - There is still work to do to bootstrap GHC until the latest version on
>   i686, and to potentially shorten the bootstrap chain (Lars-Dominik Braun).
> - OCaml could be simplified by dropping version 4.07 (Julien Lepiller).
> - After the mesa update is before the mesa update, and it looks like more
>   features can be enabled (Kaelyn Takata, John Kehayias).

When I get a chance next I'll be pulling the relevant patches locally
and building as a first check. And then we'll want this feature branch
to be created and built for wider testing.

> - Too much in Guix depends on too much else, which makes building things
>   needlessly entangled; in particular time zone data should not be referred
>   to by packages, but be loaded at runtime (Leo Famulari).
>
> All the best in your Guix endeavours,
>
> Andreas

Thanks everyone!
John

Re: Build dependency inflation (was: Re: Core-updates merge)

[bokr]

Hi,
TL;DR:

If we have 22k sources, how about expressing dependency as a
22k x 22k matrix of weights Wij and dependency by
multiplication of 22k x 1 matrix os sources variables
ordered Sj by immediate dependency first. I guess the Sj
column matrix is transposed to do the multiplication, so the
Wij j's and Sj j's match.

Start with weights of only 1 0r 0. Later maybe measures of
compile times?

The diagonal Wjj would be all ones, and counts of ones in
vertical columns Wkj would be maximal if that Sj was
directly depended on for a lot of k's

Reordering the Wjj and Sj sort of like solving simultaneous
equations and colorizing somehow to show the clusters of Wij
dependency bits vs a backgrond of zero bits might make an
interesting display. WDYT?

Any GnuPlot whizzes out there? :-)

Regards,
Bengt Richter



On +2023-04-27 19:56:38 +0200, Simon Josefsson via Development of GNU Guix and 
the GNU System distribution. wrote:
> Andreas Enge  writes:
> 
> > - Too much in Guix depends on too much else, which makes building things
> >   needlessly entangled; in particular time zone data should not be referred
> >   to by packages, but be loaded at runtime (Leo Famulari).
> 
> This is an important open problem -- is there any way to attack this
> problem in a systematic way?  I guess it is hard to understand which
> packages ends up depending on what since it is a large graph with long
> cycles, and also to understand which build depencies are essential and
> which are superficial, and thus consequently challenging to know where
> to start working to reduce build dependencies?
> 
> I wonder if it is possible to graph all the build dependencies, and do
> something like a monte-carlo what-if simulation: randomly pick one
> build-dependency from the entire build-dependency graph and remove it,
> and recompute the graph.  If the difference between these two graphs
> leads to a significantly lower total build computational cost than
> before, we may be on to something.  My theory is that "true" build
> dependencies will show up in so many places that removing just one
> instance of it will not affect total build time.  But "needless" build
> dependencies may occur just once or few times, and this approach would
> catch low-hanging fruit to work on.  Maybe the simulation needs to be
> done on more than just removing one build-dependency, it could play
> what-if games removing two build-dependencies etc, or three random
> build-dependencies, and so on.  Maybe my idea is flawed, and this will
> only lead to a list of build-dependencies that are impossible to get rid
> off anyway.
> 
> Is there some other method to understand what build dependencies would
> be important to remove, to speed up total rebuild time?
> 
> Maybe we could analyze how much of a particular build-dependency
> actually is used by a particular build?  By looking into file-access
> patterns etc.
> 
> /Simon

Build dependency inflation (was: Re: Core-updates merge)

[Development of GNU Guix and the GNU System distribution.]

Andreas Enge  writes:

> - Too much in Guix depends on too much else, which makes building things
>   needlessly entangled; in particular time zone data should not be referred
>   to by packages, but be loaded at runtime (Leo Famulari).

This is an important open problem -- is there any way to attack this
problem in a systematic way?  I guess it is hard to understand which
packages ends up depending on what since it is a large graph with long
cycles, and also to understand which build depencies are essential and
which are superficial, and thus consequently challenging to know where
to start working to reduce build dependencies?

I wonder if it is possible to graph all the build dependencies, and do
something like a monte-carlo what-if simulation: randomly pick one
build-dependency from the entire build-dependency graph and remove it,
and recompute the graph.  If the difference between these two graphs
leads to a significantly lower total build computational cost than
before, we may be on to something.  My theory is that "true" build
dependencies will show up in so many places that removing just one
instance of it will not affect total build time.  But "needless" build
dependencies may occur just once or few times, and this approach would
catch low-hanging fruit to work on.  Maybe the simulation needs to be
done on more than just removing one build-dependency, it could play
what-if games removing two build-dependencies etc, or three random
build-dependencies, and so on.  Maybe my idea is flawed, and this will
only lead to a list of build-dependencies that are impossible to get rid
off anyway.

Is there some other method to understand what build dependencies would
be important to remove, to speed up total rebuild time?

Maybe we could analyze how much of a particular build-dependency
actually is used by a particular build?  By looking into file-access
patterns etc.

/Simon


signature.asc
Description: PGP signature

Re: [Rust Team] Status post core-updates merge

[Efraim Flashner]

On Thu, Apr 27, 2023 at 07:06:29PM +0200, Josselin Poiret wrote:
> Hi Efraim (and the rest of the Rust team),
> 
> Efraim Flashner  writes:
> 
> > Now that core-updates has been merged into master it's time for a status
> > update from the rust team.
> >
> > Currently rust supports (in the order of gaining support) x86_64,
> > aarch64 and riscv64. GDB-11 doesn't currently build on aarch64 or
> > riscv64, so we're removing gdb-11 entirely and using a new gdb-12 as
> > gdb/pinned for the test suite. This is expected to fix rust building on
> > aarch64.
> 
> Is there any hope for the other architectures?  I'm thinking i686
> mostly, but s390x, powerpc and mips are all on the radar.

There hasn't been much development in the mrustc repo for the past few
months and I believe the issues blocking i686 (needs too much memory at
once) is still present. As far as powerpc64le, it does pretty well with
mrustc but doesn't get through the whole bootstrap yet.

Another option would be to work on the cross-compiling ability of the
cargo-build-system, then you could at least cross compile to them.

I will point out that s390x and mips aren't actually supported in Guix
currently.

> > rust-bootstrap currently fails to build on riscv64. I've been testing
> > solutions by building on an x86_64 machine to see what builds pass and
> > what fails, and I think I've found a solution with minimal changes.
> >
> > Once I'm able to successfully build rust-bootstrap on x86_64 and riscv64
> > I'll push the update to the rust-team branch and let the CI build it
> > out.
> >
> > As a quick overview, this push moves our rust package to rust-1.67 and
> > updates many of our rust packages. We've had some more unbundling of
> > bundled library sources and precompiled libraries and updated a number
> > of rust programs, mostly so they build with the newer version of rust.
> 
> Very impressive!  I only watch from afar and complain about the rust
> situation whenever I can, but this is giving me hope.  With Rust getting
> further and further down into the dependency chain, this is vital work,
> and you all seem to be handling it very well.
> 
> I'm also curious, since you're "early adopters": how are you managing
> with the team workflow?  Any tips, remarks or impressions that you could
> share with other teams?

Speak in the plural and no one will realize it's a team of one ;)

I'm thinking of tagging the rust stuff with [rust team] so it stands out
and is easily searchable. if/when I send patches for other teams I'll
probably tag those too and hope it catches on.


-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: [Rust Team] Status post core-updates merge

[Josselin Poiret]

Hi Efraim (and the rest of the Rust team),

Efraim Flashner  writes:

> Now that core-updates has been merged into master it's time for a status
> update from the rust team.
>
> Currently rust supports (in the order of gaining support) x86_64,
> aarch64 and riscv64. GDB-11 doesn't currently build on aarch64 or
> riscv64, so we're removing gdb-11 entirely and using a new gdb-12 as
> gdb/pinned for the test suite. This is expected to fix rust building on
> aarch64.

Is there any hope for the other architectures?  I'm thinking i686
mostly, but s390x, powerpc and mips are all on the radar.

> rust-bootstrap currently fails to build on riscv64. I've been testing
> solutions by building on an x86_64 machine to see what builds pass and
> what fails, and I think I've found a solution with minimal changes.
>
> Once I'm able to successfully build rust-bootstrap on x86_64 and riscv64
> I'll push the update to the rust-team branch and let the CI build it
> out.
>
> As a quick overview, this push moves our rust package to rust-1.67 and
> updates many of our rust packages. We've had some more unbundling of
> bundled library sources and precompiled libraries and updated a number
> of rust programs, mostly so they build with the newer version of rust.

Very impressive!  I only watch from afar and complain about the rust
situation whenever I can, but this is giving me hope.  With Rust getting
further and further down into the dependency chain, this is vital work,
and you all seem to be handling it very well.

I'm also curious, since you're "early adopters": how are you managing
with the team workflow?  Any tips, remarks or impressions that you could
share with other teams?

Best,
---
Josselin Poiret


signature.asc
Description: PGP signature

[Rust Team] Status post core-updates merge

[Efraim Flashner]

Now that core-updates has been merged into master it's time for a status
update from the rust team.

Currently rust supports (in the order of gaining support) x86_64,
aarch64 and riscv64. GDB-11 doesn't currently build on aarch64 or
riscv64, so we're removing gdb-11 entirely and using a new gdb-12 as
gdb/pinned for the test suite. This is expected to fix rust building on
aarch64.

rust-bootstrap currently fails to build on riscv64. I've been testing
solutions by building on an x86_64 machine to see what builds pass and
what fails, and I think I've found a solution with minimal changes.

Once I'm able to successfully build rust-bootstrap on x86_64 and riscv64
I'll push the update to the rust-team branch and let the CI build it
out.

As a quick overview, this push moves our rust package to rust-1.67 and
updates many of our rust packages. We've had some more unbundling of
bundled library sources and precompiled libraries and updated a number
of rust programs, mostly so they build with the newer version of rust.

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Core-updates merge

[Maxim Cournoyer]

Hello,

My gratitude for your patience and perseverance on merging the
core-updates branch :-).

Congratulations everyone!

-- 
Thanks,
Maxim

Re: Core-updates merge

[Leo Famulari]

Thank you for leading this effort!

The value of leadership and project management is sometimes underappreciated in 
software projects, but they are essential for our success.

On Tue, Apr 25, 2023, at 10:09, Andreas Enge wrote:
> Hello all,
>
> I have just merged core-updates into master and deleted the branch!
> This has been a long adventure, which became particularly intensive
> after the last Guix Days in February. First and foremost many thanks to
> everyone who contributed to the branch, be it by commits, discussions or
> by working on the infrastructure.
>
> Each and every package is not yet in shape; please feel free to submit
> patches for your favourite packages that fail to build. In particular:
> - python-yubikey-manager does not build currently; work to correct this
>   is underway.
> - R on powerpc does not build; this will also be corrected soon;
> - aarch64 has very few substitutes; I think this is mainly due to the build
>   farm catching up and not so much to packages not building, but this is
>   difficult to know.
> If any of these are essential to you, you may wish to wait a little bit
> longer before a "guix pull", or for the time being pull to a commit just
> before the merge by issuing
>guix pull --commit=472706ae2f9160833951a4e4bcc4c206e03097b0
>
> Every end of a story is the beginning of many new ones. Several areas of
> work have already been identified; I will summarise what came up on the
> mailing list and who expressed interest to work on it - please feel free
> to join if a topic interests you, or launch another initiative if you
> want to work on a different subject!
> - rust-team already has a branch that is almost ready to be built and
>   merged, as a precursor to the team based workflow that we need to invent
>   (Efraim Flashner).
> - R on powerpc64le needs to be built by changes to valgrind and lz4
>   (Simon Tournier, I).
> - Many Python packages need updates, in particular with the aim of building
>   python-yubikey-manager (John Kehayias, Lars-Dominik Braun, Brian Cully).
> - There is still work to do to bootstrap GHC until the latest version on
>   i686, and to potentially shorten the bootstrap chain (Lars-Dominik Braun).
> - OCaml could be simplified by dropping version 4.07 (Julien Lepiller).
> - After the mesa update is before the mesa update, and it looks like more
>   features can be enabled (Kaelyn Takata, John Kehayias).
> - Too much in Guix depends on too much else, which makes building things
>   needlessly entangled; in particular time zone data should not be referred
>   to by packages, but be loaded at runtime (Leo Famulari).
>
> All the best in your Guix endeavours,
>
> Andreas

Re: Core-updates merge

[Josselin Poiret]

Hi Andreas,

Andreas Enge  writes:

> Hello all,
>
> I have just merged core-updates into master and deleted the branch!
> This has been a long adventure, which became particularly intensive
> after the last Guix Days in February. First and foremost many thanks to
> everyone who contributed to the branch, be it by commits, discussions or
> by working on the infrastructure.

Great work and congrats!  Thank you for taking care of this, especially
with the time it took.  Hopefully we'll never have to do this again :)

> Each and every package is not yet in shape; please feel free to submit
> patches for your favourite packages that fail to build. In particular:
> - python-yubikey-manager does not build currently; work to correct this
>   is underway.
> - R on powerpc does not build; this will also be corrected soon;
> - aarch64 has very few substitutes; I think this is mainly due to the build
>   farm catching up and not so much to packages not building, but this is
>   difficult to know.

The graphical installer is also broken, but that should be adressed by
my recent patchset.

> Every end of a story is the beginning of many new ones. Several areas of
> work have already been identified; I will summarise what came up on the
> mailing list and who expressed interest to work on it - please feel free
> to join if a topic interests you, or launch another initiative if you
> want to work on a different subject!
> [...]

I also want to add something to the list: working out how we can best
implement the team-driven workflow.  We're currently lacking a lot of
documentation for it, and one thing I would really love to see is what
each team's current goals are.  Following the MLs/IRC is one thing, but
not everyone is able to read up on everything.  Should intra-team
communication be on guix-devel for everyone to see?  Should we display
the teams on the website?  How should we approach merging feature
branches to master?

Also, we have the new release on the horizon (I don't really remember
how far in the future it is), so we could maybe try to create a calendar
for it, and set team goals accordingly?

Best,
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Core-updates merge

[Katherine Cox-Buday]

On 4/25/23 8:40 AM, Felix Lechner via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi Andreas,

On Tue, Apr 25, 2023 at 7:09 AM Andreas Enge  wrote:


many thanks to
everyone who contributed to the branch, be it by commits, discussions or
by working on the infrastructure.


I'd like to please also acknowledge your pivotal role in shepherding
that monumental merge.

With grace and patience you took on an unrewarding legacy task that
had hardly any potential for glory. For your dedication in advancing
the codebase, you earned my sincere admiration and gratitude. Thank
you!


An emphatic +1.

Re: Core-updates merge

[Development of GNU Guix and the GNU System distribution.]

Hi Andreas,

On Tue, Apr 25, 2023 at 7:09 AM Andreas Enge  wrote:
>
> many thanks to
> everyone who contributed to the branch, be it by commits, discussions or
> by working on the infrastructure.

I'd like to please also acknowledge your pivotal role in shepherding
that monumental merge.

With grace and patience you took on an unrewarding legacy task that
had hardly any potential for glory. For your dedication in advancing
the codebase, you earned my sincere admiration and gratitude. Thank
you!

Kind regards
Felix

Core-updates merge

[Andreas Enge]

Hello all,

I have just merged core-updates into master and deleted the branch!
This has been a long adventure, which became particularly intensive
after the last Guix Days in February. First and foremost many thanks to
everyone who contributed to the branch, be it by commits, discussions or
by working on the infrastructure.

Each and every package is not yet in shape; please feel free to submit
patches for your favourite packages that fail to build. In particular:
- python-yubikey-manager does not build currently; work to correct this
  is underway.
- R on powerpc does not build; this will also be corrected soon;
- aarch64 has very few substitutes; I think this is mainly due to the build
  farm catching up and not so much to packages not building, but this is
  difficult to know.
If any of these are essential to you, you may wish to wait a little bit
longer before a "guix pull", or for the time being pull to a commit just
before the merge by issuing
   guix pull --commit=472706ae2f9160833951a4e4bcc4c206e03097b0

Every end of a story is the beginning of many new ones. Several areas of
work have already been identified; I will summarise what came up on the
mailing list and who expressed interest to work on it - please feel free
to join if a topic interests you, or launch another initiative if you
want to work on a different subject!
- rust-team already has a branch that is almost ready to be built and
  merged, as a precursor to the team based workflow that we need to invent
  (Efraim Flashner).
- R on powerpc64le needs to be built by changes to valgrind and lz4
  (Simon Tournier, I).
- Many Python packages need updates, in particular with the aim of building
  python-yubikey-manager (John Kehayias, Lars-Dominik Braun, Brian Cully).
- There is still work to do to bootstrap GHC until the latest version on
  i686, and to potentially shorten the bootstrap chain (Lars-Dominik Braun).
- OCaml could be simplified by dropping version 4.07 (Julien Lepiller).
- After the mesa update is before the mesa update, and it looks like more
  features can be enabled (Kaelyn Takata, John Kehayias).
- Too much in Guix depends on too much else, which makes building things
  needlessly entangled; in particular time zone data should not be referred
  to by packages, but be loaded at runtime (Leo Famulari).

All the best in your Guix endeavours,

Andreas

Core-updates merge, d-1

[Andreas Enge]

Hello,

people have been working on packages close to the leaves which did not
build any more in core-updates; but as far as I can see, nothing major
has popped up that would prevent a merge.

So unless there is firm opposition, I intend to merge core-updates to
master tomorrow as announced, in the early European afternoon.

Andreas

Re: package systems, before and after the core-updates merge

[zimoun]

Hi,

On Sat, 19 Oct 2019 at 23:03, Ludovic Courtès  wrote:

> > I think this is down to the use of package-transitive-supported-systems
> > within the Guix Data Service, but the output of this function for a
> > package can also be seen by running guix package --show.
>
> The patch below fixes ‘guix show’ but it has a noticeable performance
> impact that makes me thing something’s not quite right with memoization
> in ‘package-transitive-supported-systems’:

How the performances can be tested?


All the best,
simon

Re: package systems, before and after the core-updates merge

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> One thing that I was aware of before the recent core-updates merge was
> that the Guix Data Service [1] didn't generate derivations for systems
> other than x86_64-linux and i686-linux, at least to the same extent as
> the master branch before the recent core-updates merge.

Indeed, see this bug fix that landed before the merge:

  
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=bc60349b5bc58a0b803df5adce1de6db82453744

> I think this is down to the use of package-transitive-supported-systems
> within the Guix Data Service, but the output of this function for a
> package can also be seen by running guix package --show.

The patch below fixes ‘guix show’ but it has a noticeable performance
impact that makes me thing something’s not quite right with memoization
in ‘package-transitive-supported-systems’:

diff --git a/guix/ui.scm b/guix/ui.scm
index 3e4bd5787e..426e517b54 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -1259,7 +1259,8 @@ WIDTH columns.  EXTRA-FIELDS is a list of symbol/value pairs to emit."
   (format port "version: ~a~%" (package-version p))
   (format port "outputs: ~a~%" (string-join (package-outputs p)))
   (format port "systems: ~a~%"
-  (string-join (package-transitive-supported-systems p)))
+  (string-join (filter (cut supported-package? p <>)
+   %supported-systems)))
   (format port "dependencies: ~a~%"
   (match (package-direct-inputs p)
 (((labels inputs . _) ...)

> Also, for the Guix Data Service, all I want to know is for a given
> package, is for which systems and targets a derivation can be reasonably
> computed. Maybe it is wrong to use package-transitive-supported-systems
> for this.

You can use ‘supported-package?’ as above.  This is also what (gnu ci)
does.

Let me know if this helps!

Ludo’.

package systems, before and after the core-updates merge

[Christopher Baines]

Hey,

One thing that I was aware of before the recent core-updates merge was
that the Guix Data Service [1] didn't generate derivations for systems
other than x86_64-linux and i686-linux, at least to the same extent as
the master branch before the recent core-updates merge.

1: http://data.guix.gnu.org/revision/2fa55c72476c73211cbb2d6b29c05a1ad58a6cf9
   (see the Derivations table)

I put this down to a bug I wasn't seeing, but now that the core-updates
branch has been merged, and now that this effect is showing up on the
master branch, I've investigated a little more.

I think this is down to the use of package-transitive-supported-systems
within the Guix Data Service, but the output of this function for a
package can also be seen by running guix package --show.

Before the recent core-updates merge:

  ./pre-inst-env guix package --show=hello
  …
  systems: x86_64-linux i686-linux armhf-linux aarch64-linux mips64el-linux

After the recent core-updates merge:

  → guix package --show=hello
  …
  systems: x86_64-linux i686-linux

Looking at the implementation of package-transitive-supported-systems,
I'm pretty sure the change in behaviour is to do with the
supported-systems for the new packages involved in the bootstrap
process.

So in terms of questions I now have, what's the "systems: " output by
guix package --show meant to mean, and what does the recent change
removing the 3 systems above from most packages in Guix mean?

Also, for the Guix Data Service, all I want to know is for a given
package, is for which systems and targets a derivation can be reasonably
computed. Maybe it is wrong to use package-transitive-supported-systems
for this.

Thanks,

Chris


signature.asc
Description: PGP signature

Re: packages which are broken after core-updates merge

[Ricardo Wurmus]

Ludovic Courtès  writes:

> ng0  skribis:
>
>> icecat is failing, with new profiles, with old profiles. This can be
>> reproduced in a new profile by selecting 'Preferences > Applications'.
>> It should immediately crash.
>
> I have /gnu/store/pk4sbzk3xh7yjj9ls7hkhzgaj0xha7xp-icecat-45.7.0-gnu1
> (x86_64) and I can reproduce the Preferences → Applications crash.
> However, it does not crash immediately upon startup as Ricardo
> experiences.

I use the very same variant of icecat.  (Having unique names is really
useful for debugging!)

I have a large profile with many tabs.  When icecat starts it restores
the tabs and crashes before I can even see anything.  When I move away
my profile and start in “safe mode” the window appears and Icecat
crashes when it tries to render some URL.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: packages which are broken after core-updates merge

[Clément Lassieur]

Ricardo Wurmus  writes:

> Clément Lassieur  writes:
>
>>> Clément Lassieur transcribed 0.2K bytes:
 > icecat is failing, with new profiles, with old profiles. This can be
 > reproduced in a new profile by selecting 'Preferences > Applications'.
 > It should immediately crash.

 It doesn't crash when "--with-system-icu" is removed.
>>>
>>> Can you send a patch for this?
>>
>> Sure, but I'm not sure it is the right solution.  It would be preferable
>> to use system libraries when we can.  What the other think?
>
> This usually indicates that upstream forked the library (or that the
> depend on this particular version of the API).  I think it’s better to
> comment the configure flag with a FIXME and have a version of Icecat
> that is usable than a version of Icecat that immediately crashes when
> you start it.
>
> In any case I think it would be good to file a bug report upstream with
> information about the exact version of ICU that we’re building with.

See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26361.

Re: packages which are broken after core-updates merge

[Clément Lassieur]

> Clément Lassieur  writes:
>
>>> Clément Lassieur transcribed 0.2K bytes:
 > icecat is failing, with new profiles, with old profiles. This can be
 > reproduced in a new profile by selecting 'Preferences > Applications'.
 > It should immediately crash.

 It doesn't crash when "--with-system-icu" is removed.
>>>
>>> Can you send a patch for this?
>>
>> Sure, but I'm not sure it is the right solution.  It would be preferable
>> to use system libraries when we can.  What the other think?
>
> This usually indicates that upstream forked the library (or that the
> depend on this particular version of the API).  I think it’s better to
> comment the configure flag with a FIXME and have a version of Icecat
> that is usable than a version of Icecat that immediately crashes when
> you start it.
>
> In any case I think it would be good to file a bug report upstream with
> information about the exact version of ICU that we’re building with.

Alright, I'll send a patch then.  Thanks for commenting.

Re: packages which are broken after core-updates merge

[Clément Lassieur]

> ng0  skribis:
>
>> icecat is failing, with new profiles, with old profiles. This can be
>> reproduced in a new profile by selecting 'Preferences > Applications'.
>> It should immediately crash.
>
> I have /gnu/store/pk4sbzk3xh7yjj9ls7hkhzgaj0xha7xp-icecat-45.7.0-gnu1
> (x86_64) and I can reproduce the Preferences → Applications crash.
> However, it does not crash immediately upon startup as Ricardo
> experiences.

It also crashes everytime when browsing "about:support".  Immediate
crash depends on the profile I think.

Re: packages which are broken after core-updates merge

[Ludovic Courtès]

ng0  skribis:

> icecat is failing, with new profiles, with old profiles. This can be
> reproduced in a new profile by selecting 'Preferences > Applications'.
> It should immediately crash.

I have /gnu/store/pk4sbzk3xh7yjj9ls7hkhzgaj0xha7xp-icecat-45.7.0-gnu1
(x86_64) and I can reproduce the Preferences → Applications crash.
However, it does not crash immediately upon startup as Ricardo
experiences.

Ludo’.

Re: packages which are broken after core-updates merge

[Ricardo Wurmus]

Clément Lassieur  writes:

>> Clément Lassieur transcribed 0.2K bytes:
>>> > icecat is failing, with new profiles, with old profiles. This can be
>>> > reproduced in a new profile by selecting 'Preferences > Applications'.
>>> > It should immediately crash.
>>>
>>> It doesn't crash when "--with-system-icu" is removed.
>>
>> Can you send a patch for this?
>
> Sure, but I'm not sure it is the right solution.  It would be preferable
> to use system libraries when we can.  What the other think?

This usually indicates that upstream forked the library (or that the
depend on this particular version of the API).  I think it’s better to
comment the configure flag with a FIXME and have a version of Icecat
that is usable than a version of Icecat that immediately crashes when
you start it.

In any case I think it would be good to file a bug report upstream with
information about the exact version of ICU that we’re building with.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: packages which are broken after core-updates merge

[Clément Lassieur]

> Clément Lassieur transcribed 0.2K bytes:
>> > icecat is failing, with new profiles, with old profiles. This can be
>> > reproduced in a new profile by selecting 'Preferences > Applications'.
>> > It should immediately crash.
>> 
>> It doesn't crash when "--with-system-icu" is removed.
>
> Can you send a patch for this?

Sure, but I'm not sure it is the right solution.  It would be preferable
to use system libraries when we can.  What the other think?

Re: packages which are broken after core-updates merge

[ng0]

Clément Lassieur transcribed 0.2K bytes:
> > icecat is failing, with new profiles, with old profiles. This can be
> > reproduced in a new profile by selecting 'Preferences > Applications'.
> > It should immediately crash.
> 
> It doesn't crash when "--with-system-icu" is removed.

Can you send a patch for this?

Re: packages which are broken after core-updates merge

[Clément Lassieur]

> icecat is failing, with new profiles, with old profiles. This can be
> reproduced in a new profile by selecting 'Preferences > Applications'.
> It should immediately crash.

It doesn't crash when "--with-system-icu" is removed.

packages which are broken after core-updates merge

[ng0]

neomutt no longer works with kyotocabinet correctly:
Apr  3 13:52:29 localhost vmunix: [  426.159873] traps: mutt[937] trap
invalid opcode ip:7f15d1356008 sp:7fffe89d9070 error:0
Apr  3 13:52:29 localhost vmunix: [  426.159876]  in
libkyotocabinet.so.16.13.0[7f15d12e9000+ee000]

icecat is failing, with new profiles, with old profiles. This can be
reproduced in a new profile by selecting 'Preferences > Applications'.
It should immediately crash.

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Sat 06 Aug 2016 09:52, Andreas Enge  writes:

> I am not very happy about such a policy; if I sign a commit, I am only
> signing my commit, and not all of its history

For what it's worth, this is not quite true.  The SHA1 hashes of parent
commits are part of the commit object itself, so if you sign a commit
you are also signing its history.

Andy

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Sat 06 Aug 2016 04:07, Leo Famulari  writes:

> But, I also think the primary point of signing the commits is to record
> the identity of the person responsible for the commit, and so I think
> the policy should be to sign each commit. [0]

To me this is not the value that signing brings; rather, signing
protects against an attack in which a malicious third party updates the
Guix git repository to have a vulnerable commit.

Given that most people run "guix pull" without inspecting the commits,
this is real value: it would be possible to even make "guix pull" only
accept updates whose HEAD is signed by a key in the keyring.  Having the
hook only accept signed HEADs is a good start along that path of course.

> Isn't it better for the identity information to be inherent to the Git
> commits themselves, since those are what is preserved by Git? Git does
> not preserve hooks or policies.

The convention that a signature goes along with responsibility is also a
policy -- any path we take is a convention.

> Also, is there some problem with signing each commit? I don't know why
> we'd want to stop doing this.

I think there's a risk of signing fatigue.  The more signatures you make
with your key, the more likely it is that you sign something that you
didn't mean to.  To me it makes sense to reduce the number of signatures
to the minimum necessary to preserve whatever security properties we are
interested in; but YMMV obviously :)

Andy

Re: ‘core-updates’ merge is a squashed commit

[Mike Gerwitz]

On Thu, Aug 04, 2016 at 17:06:15 +0200, Andy Wingo wrote:
> What's the rationale for requiring non-HEAD commits to be signed when
> pushing?  For me a signed HEAD implicitly signs all parent comments, in
> my mental trust model anyway :)

That could be a potentially daunting/impossible task for the person
signing a commit.

Aside from asserting one's identity, GPG-signed commits also (can) help
in the event that the system of one of the Guix hackers with commit
access is compromised.  Attacking Savannah is one way to compromise the
repo, but compromising one of the many Guix hackers' systems is another.

If a commit is signed in the hacker's local repo, it cannot be
manipulated by an attacker, nor can an attacker sign a new malicious
commit.  Unless, of course, the GPG key resides on the same box, the
attacker can get a hold of it, and can use a keylogger/etc to get the
passphrase.  Smart cards help here.

I also recommend against auto-signing commmits on rebase unless you
first verify that each commit within that range has a valid signature
beforehand.

Not fool-proof, but nothing is. :)

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
https://mikegerwitz.com   | GPG Key ID: 0x8EE30EAB


signature.asc
Description: PGP signature

Re: ‘core-updates’ merge is a squashed commit

[Andreas Enge]

On Fri, Aug 05, 2016 at 08:59:32PM -0400, Mark H Weaver wrote:
> I haven't thought deeply on this, but it seems to me that Andy's
> suggestion has a lot of merit.  We could choose to decide, as a matter
> of policy, that if you sign a commit with unsigned ancestor commit(s),
> you are effectively vouching for those ancestor commits.  We could
> modify the commit hook to accept a push as long as the new HEAD commit
> is signed by an authorized key, disregarding the ancestors.
> 
> There's one thing that each of us would need to be careful of, though.
> If we adopt this policy, then before signing a commit, we'd need to
> first verify that the parent commit has been signed, lest we
> accidentally vouch for an unsigned commit that we know nothing about.

I am not very happy about such a policy; if I sign a commit, I am only
signing my commit, and not all of its history, or even only its history
up to the previous signed commit. Also, while signing each commit is
a simple git configuration option, needing to verify the history before
each commit would be a hassle that as far as I can see is not easily
automated.

> In practice, this could only happen if Savannah is compromised or
> there's a man-in-the-middle attack, because Savannah is supposed to
> ensure that pushes with unsigned HEADs are rejected.

Agreed, this mitigates the problem above. But I feel better with the
current situation.

Andreas

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Fri, Aug 05, 2016 at 08:59:32PM -0400, Mark H Weaver wrote:
> Leo Famulari  writes:
> > If I push A-B-C with a signed HEAD immediately after somebody pushes a
> > forged D, won't it look like I vouch for D?
> 
> This can't happen unless D is already in your local repo before you
> commit locally.  If someone pushes D immediately before you try to push
> A-B-C, your push will be rejected.  At that point, you need to pull,
> rebase on top of D, and try again.  You can always see the ancestor
> commits before signing the new comit.
> 
> I haven't thought deeply on this, but it seems to me that Andy's
> suggestion has a lot of merit.  We could choose to decide, as a matter
> of policy, that if you sign a commit with unsigned ancestor commit(s),
> you are effectively vouching for those ancestor commits.  We could
> modify the commit hook to accept a push as long as the new HEAD commit
> is signed by an authorized key, disregarding the ancestors.
> 
> There's one thing that each of us would need to be careful of, though.
> If we adopt this policy, then before signing a commit, we'd need to
> first verify that the parent commit has been signed, lest we
> accidentally vouch for an unsigned commit that we know nothing about.
> 
> In practice, this could only happen if Savannah is compromised or
> there's a man-in-the-middle attack, because Savannah is supposed to
> ensure that pushes with unsigned HEADs are rejected.
>
> What do you think?

Okay, I think your analysis is right. If the HEAD of a push is always
signed, and that policy is known, then it is possible to know who is
responsible for pushing each commit.

But, I also think the primary point of signing the commits is to record
the identity of the person responsible for the commit, and so I think
the policy should be to sign each commit. [0]

The identity information provided by a "sign the HEAD" system seems
relatively brittle to me.

Somebody looking at a clone of the Git repo may not know about a commit
hook, or a "be careful" policy, that may or may not have been
consistently active (it seems that hooks don't come with `git clone`).
This person won't be able to easily attribute the unsigned commits.

Isn't it better for the identity information to be inherent to the Git
commits themselves, since those are what is preserved by Git? Git does
not preserve hooks or policies.

Also, is there some problem with signing each commit? I don't know why
we'd want to stop doing this.

[0] But, I also think that the hook we put in place to enforce this is
going to have to be modified, at least until we no longer need to push
this last batch of unsigned commits. At the very least, it should be
made to actually verify the signatures.

Re: ‘core-updates’ merge is a squashed commit

[Mark H Weaver]

Leo Famulari  writes:

> On Fri, Aug 05, 2016 at 06:50:30PM +0200, Andy Wingo wrote:
>> Why would you sign a commit if you don't attest to intermediate unsigned
>> commits?
>
> If I push A-B-C with a signed HEAD immediately after somebody pushes a
> forged D, won't it look like I vouch for D?

This can't happen unless D is already in your local repo before you
commit locally.  If someone pushes D immediately before you try to push
A-B-C, your push will be rejected.  At that point, you need to pull,
rebase on top of D, and try again.  You can always see the ancestor
commits before signing the new comit.

I haven't thought deeply on this, but it seems to me that Andy's
suggestion has a lot of merit.  We could choose to decide, as a matter
of policy, that if you sign a commit with unsigned ancestor commit(s),
you are effectively vouching for those ancestor commits.  We could
modify the commit hook to accept a push as long as the new HEAD commit
is signed by an authorized key, disregarding the ancestors.

There's one thing that each of us would need to be careful of, though.
If we adopt this policy, then before signing a commit, we'd need to
first verify that the parent commit has been signed, lest we
accidentally vouch for an unsigned commit that we know nothing about.

In practice, this could only happen if Savannah is compromised or
there's a man-in-the-middle attack, because Savannah is supposed to
ensure that pushes with unsigned HEADs are rejected.

What do you think?

  Mark

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Fri, Aug 05, 2016 at 06:50:30PM +0200, Andy Wingo wrote:
> Why would you sign a commit if you don't attest to intermediate unsigned
> commits?

If I push A-B-C with a signed HEAD immediately after somebody pushes a
forged D, won't it look like I vouch for D? How could a 3rd party tell
whether D was pushed by me or somebody else?

Does your suggested method address this hypothetical situation?

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Fri 05 Aug 2016 16:59, Leo Famulari  writes:

> On Fri, Aug 05, 2016 at 09:35:59AM +0200, Andy Wingo wrote:
>> Yeah.  I guess I don't see see "author misattribution on unsigned
>> commits" as part of the threat model.
>> 
>> My mental model is that if you have a signed commit A with unsigned
>> parents B, C, ..., that it's the person who signed commit A who signs
>> off on commits B, C, and so on.  That person attests to the integrity of
>> that range of commits, *including* the author field(s).
>
> But, how does anyone know that the person who signed A attests to B and
> C? I don't think Git has a feature that conveys that intention.

Why would you sign a commit if you don't attest to intermediate unsigned
commits?

A

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Fri, Aug 05, 2016 at 09:35:59AM +0200, Andy Wingo wrote:
> Yeah.  I guess I don't see see "author misattribution on unsigned
> commits" as part of the threat model.
> 
> My mental model is that if you have a signed commit A with unsigned
> parents B, C, ..., that it's the person who signed commit A who signs
> off on commits B, C, and so on.  That person attests to the integrity of
> that range of commits, *including* the author field(s).

But, how does anyone know that the person who signed A attests to B and
C? I don't think Git has a feature that conveys that intention.

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Thu 04 Aug 2016 22:05, Leo Famulari  writes:

> On Thu, Aug 04, 2016 at 06:55:34PM +0200, Andy Wingo wrote:
>> On Thu 04 Aug 2016 18:44, Leo Famulari  writes:
>> 
>> > How would the rest of us distinguish between
>> >
>> > 1) a range of your commits with a signed HEAD
>> > 2) a range of your commits with a signed HEAD that you pushed after I
>> > pushed a commit created with `git commit --author="Andy Wingo"
>> 
>> I'm not sure what the threat model here is, and surely this is mostly
>> because I am ignorant :)  Would you mind elaborating a bit more?
>
> I admit, the example is really contrived.
>
> My point is that, as far as I know, there is no way to know who exactly
> is behind an unsigned Git commit.
>
> The "Author" and "Commit" information seen in `git log --format=full` is
> trivially forged, for example by altering the [user] field of your Git
> configuration file.

Yeah.  I guess I don't see see "author misattribution on unsigned
commits" as part of the threat model.

My mental model is that if you have a signed commit A with unsigned
parents B, C, ..., that it's the person who signed commit A who signs
off on commits B, C, and so on.  That person attests to the integrity of
that range of commits, *including* the author field(s).

If you sign a HEAD which brings in an unsigned commit that you (or
someone else) forged to use me (say) as --author, it's true, I can claim
not to have made it.  But that seems a bit irrelevant to any property we
care about; dunno...

Andy

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Thu, Aug 04, 2016 at 06:55:34PM +0200, Andy Wingo wrote:
> On Thu 04 Aug 2016 18:44, Leo Famulari  writes:
> 
> > How would the rest of us distinguish between
> >
> > 1) a range of your commits with a signed HEAD
> > 2) a range of your commits with a signed HEAD that you pushed after I
> > pushed a commit created with `git commit --author="Andy Wingo"
> 
> I'm not sure what the threat model here is, and surely this is mostly
> because I am ignorant :)  Would you mind elaborating a bit more?

I admit, the example is really contrived.

My point is that, as far as I know, there is no way to know who exactly
is behind an unsigned Git commit.

The "Author" and "Commit" information seen in `git log --format=full` is
trivially forged, for example by altering the [user] field of your Git
configuration file.

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Thu, Aug 04, 2016 at 08:32:42PM +0200, Andreas Enge wrote:
> On Thu, Aug 04, 2016 at 12:37:07PM -0400, Leo Famulari wrote:
> > git rebase "$range" --exec "git commit --amend --no-edit --gpg-sign" || 
> > git rebase --abort
> 
> So it is not enough to just do a
>git rebase -S
> ? I thought this would re-sign all my commits on top of the base of the 
> rebase.
> If not, this could explain my earlier mistake...

I don't remember exactly why I began using the `git rebase --exec ...`
command. If yours works, I'd like to switch to it!

Re: ‘core-updates’ merge is a squashed commit

[Andreas Enge]

On Thu, Aug 04, 2016 at 04:45:56PM +0200, Mathieu Lirzin wrote:
> With gpg-agent and git properly setup, signing every local commit is not
> that inconvenient IME.
> I recommend you to give a try.  ;)

Ever so often I try gpg@2 (I do not remember whether 2.0 or 2.1), it works
for a while, it stops working, and then I get back to @1...

Andreas

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Thu 04 Aug 2016 18:44, Leo Famulari  writes:

> How would the rest of us distinguish between
>
> 1) a range of your commits with a signed HEAD
> 2) a range of your commits with a signed HEAD that you pushed after I
> pushed a commit created with `git commit --author="Andy Wingo"

I'm not sure what the threat model here is, and surely this is mostly
because I am ignorant :)  Would you mind elaborating a bit more?

Andy

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Thu, Aug 04, 2016 at 05:06:15PM +0200, Andy Wingo wrote:
> On Thu 04 Aug 2016 14:36, Mark H Weaver  writes:
> 
> > Unfortunately, when I tried to push it to 'master', it was rejected
> > because of 56 unsigned commits on the 'core-updates' branch.
> 
> What's the rationale for requiring non-HEAD commits to be signed when
> pushing?  For me a signed HEAD implicitly signs all parent comments, in
> my mental trust model anyway :)

How would the rest of us distinguish between

1) a range of your commits with a signed HEAD
2) a range of your commits with a signed HEAD that you pushed after I
pushed a commit created with `git commit --author="Andy Wingo"

Re: ‘core-updates’ merge is a squashed commit

[Andy Wingo]

On Thu 04 Aug 2016 14:36, Mark H Weaver  writes:

> Unfortunately, when I tried to push it to 'master', it was rejected
> because of 56 unsigned commits on the 'core-updates' branch.

What's the rationale for requiring non-HEAD commits to be signed when
pushing?  For me a signed HEAD implicitly signs all parent comments, in
my mental trust model anyway :)

Andy

Re: ‘core-updates’ merge is a squashed commit

[Mathieu Lirzin]

Hi,

Andreas Enge  writes:

> On Thu, Aug 04, 2016 at 09:23:50AM -0400, Mark H Weaver wrote:
>> I just did this.  Huge thanks to Lisa Marie Maginnis, GNU sysadmin
>> extraordinaire, for promptly responding to my plea for help :)
>
> Thanks to both of you!
>
>> However, we should take steps to avoid this problem in the future.  The
>> most recent unsigned commit on 'core-updates' was from only 3 days ago,
>> by Andreas.  "gnu: unison: Add input ghostscript".  I'm fairly sure this
>> was after we added the commit hook, so I guess the hook only applies to
>> 'master'.
>> So, if I understand correctly, we're currently in a state where unsigned
>> commits can accidentally get pushed to other branches, making those
>> branches unmergeable into 'master'.
>
> Sorry for that! I did not think it would be possible, and working locally
> without signing every private commit is much more convenient.

With gpg-agent and git properly setup, signing every local commit is not
that inconvenient IME.

I recommend you to give a try.  ;)

-- 
Mathieu Lirzin

Re: ‘core-updates’ merge is a squashed commit

[Ludovic Courtès]

Mark H Weaver  skribis:

> Leo Famulari  writes:
>
>> On Thu, Aug 04, 2016 at 02:40:32PM +0200, Andreas Enge wrote:
>>> On Thu, Aug 04, 2016 at 08:36:28AM -0400, Mark H Weaver wrote:
>>> Good news, thanks for all this work!
>>
>> Indeed!
>>
>>> > Unfortunately, when I tried to push it to 'master', it was rejected
>>> > because of 56 unsigned commits on the 'core-updates' branch.  Note that
>>> > all of the new commits I made (the reverts, the merge, and the
>>> > cherry-picks) were all signed by me.  The unsigned commits that are
>>> > blocking this have been on the 'core-updates' branch for quite some
>>> > time.
>>> > 
>>> > If there are unsigned commits on 'core-updates-next', I guess we'll run
>>> > into the same problem trying to merge that branch later.
>>
>> Yes, I had this problem yesterday.
>>
>>> A possible solution would be to disable the commit hook that forces
>>> signatures, make these commits, and reenable the hook; hoping that noone
>>> adds an unsigned commit in the meantime... It would require to coordinate
>>> with a savannah administrator.
>>
>> I think we have to do this, unfortunately.
>
> I just did this.  Huge thanks to Lisa Marie Maginnis, GNU sysadmin
> extraordinaire, for promptly responding to my plea for help :)

Awesome, thank you all!

> However, we should take steps to avoid this problem in the future.  The
> most recent unsigned commit on 'core-updates' was from only 3 days ago,
> by Andreas.  "gnu: unison: Add input ghostscript".  I'm fairly sure this
> was after we added the commit hook, so I guess the hook only applies to
> 'master'.
>
> So, if I understand correctly, we're currently in a state where unsigned
> commits can accidentally get pushed to other branches, making those
> branches unmergeable into 'master'.

The hook can be seen at (search for “Attached Files”):

  https://savannah.gnu.org/support/?109104

I think it should work equally well for all branches.  That said, the
hook is so simple that it may well be simplistic, so do not hesitate to
send improvements to the hook in this Savannah ticket.

Thanks!

Ludo’.

Re: ‘core-updates’ merge is a squashed commit

[Mark H Weaver]

Leo Famulari  writes:

> On Thu, Aug 04, 2016 at 02:40:32PM +0200, Andreas Enge wrote:
>> On Thu, Aug 04, 2016 at 08:36:28AM -0400, Mark H Weaver wrote:
>> Good news, thanks for all this work!
>
> Indeed!
>
>> > Unfortunately, when I tried to push it to 'master', it was rejected
>> > because of 56 unsigned commits on the 'core-updates' branch.  Note that
>> > all of the new commits I made (the reverts, the merge, and the
>> > cherry-picks) were all signed by me.  The unsigned commits that are
>> > blocking this have been on the 'core-updates' branch for quite some
>> > time.
>> > 
>> > If there are unsigned commits on 'core-updates-next', I guess we'll run
>> > into the same problem trying to merge that branch later.
>
> Yes, I had this problem yesterday.
>
>> A possible solution would be to disable the commit hook that forces
>> signatures, make these commits, and reenable the hook; hoping that noone
>> adds an unsigned commit in the meantime... It would require to coordinate
>> with a savannah administrator.
>
> I think we have to do this, unfortunately.

I just did this.  Huge thanks to Lisa Marie Maginnis, GNU sysadmin
extraordinaire, for promptly responding to my plea for help :)

However, we should take steps to avoid this problem in the future.  The
most recent unsigned commit on 'core-updates' was from only 3 days ago,
by Andreas.  "gnu: unison: Add input ghostscript".  I'm fairly sure this
was after we added the commit hook, so I guess the hook only applies to
'master'.

So, if I understand correctly, we're currently in a state where unsigned
commits can accidentally get pushed to other branches, making those
branches unmergeable into 'master'.

  Mark

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Thu, Aug 04, 2016 at 02:40:32PM +0200, Andreas Enge wrote:
> On Thu, Aug 04, 2016 at 08:36:28AM -0400, Mark H Weaver wrote:
> Good news, thanks for all this work!

Indeed!

> > Unfortunately, when I tried to push it to 'master', it was rejected
> > because of 56 unsigned commits on the 'core-updates' branch.  Note that
> > all of the new commits I made (the reverts, the merge, and the
> > cherry-picks) were all signed by me.  The unsigned commits that are
> > blocking this have been on the 'core-updates' branch for quite some
> > time.
> > 
> > If there are unsigned commits on 'core-updates-next', I guess we'll run
> > into the same problem trying to merge that branch later.

Yes, I had this problem yesterday.

> A possible solution would be to disable the commit hook that forces
> signatures, make these commits, and reenable the hook; hoping that noone
> adds an unsigned commit in the meantime... It would require to coordinate
> with a savannah administrator.

I think we have to do this, unfortunately.

Re: ‘core-updates’ merge is a squashed commit

[Mark H Weaver]

Andreas Enge  writes:

> On Thu, Aug 04, 2016 at 03:50:51AM -0400, Mark H Weaver wrote:
>> How about reverting the squashed commit and then re-doing a proper merge
>> from 'core-updates-2016-08-01' into 'master'?
>
> If this is possible without too many complications, that sounds like a good
> option; the earlier the better, probably, before there are too many new
> conflicts. Hm, I just tried the revert part, and it already creates a 
> conflict.

On my local machine, I did this successfully by first reverting a couple
of later commits, then reverting the squashed merge, then doing a proper
merge, and finally cherry-picking the later commits.  I verified that
this sequence of operations made no changes to the tree.

Unfortunately, when I tried to push it to 'master', it was rejected
because of 56 unsigned commits on the 'core-updates' branch.  Note that
all of the new commits I made (the reverts, the merge, and the
cherry-picks) were all signed by me.  The unsigned commits that are
blocking this have been on the 'core-updates' branch for quite some
time.

If there are unsigned commits on 'core-updates-next', I guess we'll run
into the same problem trying to merge that branch later.

   Mark

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Thu, Aug 04, 2016 at 03:50:51AM -0400, Mark H Weaver wrote:
> How about reverting the squashed commit and then re-doing a proper merge
> from 'core-updates-2016-08-01' into 'master'?

I think we should do this. It's worth a little blemish in the commit
history in order to retain the history of this core-updates cycle.

Re: ‘core-updates’ merge is a squashed commit

[Andreas Enge]

On Thu, Aug 04, 2016 at 03:50:51AM -0400, Mark H Weaver wrote:
> How about reverting the squashed commit and then re-doing a proper merge
> from 'core-updates-2016-08-01' into 'master'?

If this is possible without too many complications, that sounds like a good
option; the earlier the better, probably, before there are too many new
conflicts. Hm, I just tried the revert part, and it already creates a conflict.

Andreas

Re: ‘core-updates’ merge is a squashed commit

[Mark H Weaver]

Leo Famulari <l...@famulari.name> writes:

> On Wed, Aug 03, 2016 at 10:29:21PM +0200, Ludovic Courtès wrote:
>> In fact, commit 455859a50f88f625d13fc2f304111f02369b366b, which is the
>> core-updates merge, is *not* a merge commit.  Instead it seems to be a
>> squashed commit of all of core-updates.  Consequently, part of the
>> history of the files touched by this merge is squashed into this single
>> pseudo-merge commit.  :-/
>
> Dang, that means we lost the commit history of core-updates. It won't be
> in the Git log.

For now, I pushed my most recent copy of the lost 'core-updates' branch
to Savannah as 'core-updates-2016-08-01', to make sure it's not lost.

>> This can be fixed locally with a graft to give the merge commit the two
>> parents it is supposed to have:
>> 
>>   git replace --graft 455859a50f88f625d13fc2f304111f02369b366b \
>>  742effef5629667b274087adc70b06abab86b252 
>> a8cb87abe98d57fb763d5b14524dc32c96bd31b5 

How about reverting the squashed commit and then re-doing a proper merge
from 'core-updates-2016-08-01' into 'master'?

  Mark

Re: ‘core-updates’ merge is a squashed commit

[Leo Famulari]

On Wed, Aug 03, 2016 at 10:29:21PM +0200, Ludovic Courtès wrote:
> In fact, commit 455859a50f88f625d13fc2f304111f02369b366b, which is the
> core-updates merge, is *not* a merge commit.  Instead it seems to be a
> squashed commit of all of core-updates.  Consequently, part of the
> history of the files touched by this merge is squashed into this single
> pseudo-merge commit.  :-/

Dang, that means we lost the commit history of core-updates. It won't be
in the Git log.

> This can be fixed locally with a graft to give the merge commit the two
> parents it is supposed to have:
> 
>   git replace --graft 455859a50f88f625d13fc2f304111f02369b366b \
>  742effef5629667b274087adc70b06abab86b252 
> a8cb87abe98d57fb763d5b14524dc32c96bd31b5 

I tried this and Git said "fatal: could not parse 
a8cb87abe98d57fb763d5b14524dc32c96bd31b5"

I don't have a Git object of that name in my repo.

‘core-updates’ merge is a squashed commit

[Ludovic Courtès]

Hello!

Someone reported that the commit stats for the new release were
suspicious and found that the recent core-updates merge was fishy.

In fact, commit 455859a50f88f625d13fc2f304111f02369b366b, which is the
core-updates merge, is *not* a merge commit.  Instead it seems to be a
squashed commit of all of core-updates.  Consequently, part of the
history of the files touched by this merge is squashed into this single
pseudo-merge commit.  :-/

Apologies for this mess.  I don’t know how this happened.  I suspect a
mixture of me not paying enough attention, and Magit + Git + gpg somehow
leading me in the wrong direction.

This can be fixed locally with a graft to give the merge commit the two
parents it is supposed to have:

  git replace --graft 455859a50f88f625d13fc2f304111f02369b366b \
 742effef5629667b274087adc70b06abab86b252 
a8cb87abe98d57fb763d5b14524dc32c96bd31b5 

Unfortunately, grafts are local, so anyone cloning the repo will see the
squashed merge commit.  Also, the replacement commit lacks a signature.

I don’t know how this impacts the core-updates-next rebase that Leo and
I were discussing.

If anyone has advice on grafts, or on how to avoid this in the future,
I’m all ears!

Ludo’.


signature.asc
Description: PGP signature

Re: Preparing for core-updates merge

[Ludovic Courtès]

Mark merged ‘core-updates’ yesterday (thanks!).
Please report any issues.

Ludo’.

Re: Preparing for core-updates merge

[Ludovic Courtès]

It’s building now!

It may be that some of your favorite packages will fail to build with
4.9, so keep an eye on it.

Ludo’.

Preparing for core-updates merge

[Ludovic Courtès]

Hello!

I think we can soon get core-updates built entirely so we can merge it
afterwards.

Any objections?

Can someone confirm that everything is OK on armhf?  (Mark tested a few
days ago, so I guess this should be OK.)

Thanks,
Ludo’.

core-updates merge really imminent

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

 If there are no objections, we’ll let Hydra build all of ‘core-updates’
 and merge it afterwards–i.e., within 2-3 days.

Ahem, sometimes, days last real long.

But we’re getting there!  http://hydra.gnu.org/jobset/gnu/core-updates
shows that most things are built.  The remaining noticeable failures
are: Qt 5 (6 packages depend on it; Qt 4 is fine), Numpy, and a few MIPS
issues.

  http://hydra.gnu.org/eval/103845#tabs-still-fail

Yesterday, Mark found a bug in the new ld-wrapper whereby, when used in
a user profile (via the ‘gcc-toolchain’ package), it would incorrectly
flag ~/.guix-profile/lib/libfoo.so as “impure.”

To work around that ‘gcc-toolchain’ is now using a fixed ld-wrapper
(commit 77db91a), meaning that we don’t have to rebuild the world right
away (although it would be good to do so for the release IMO.)

Long story short: I’ll merge tomorrow if there are no strong objections.
Please fix the remaining packages!

Thanks,
Ludo’.

core-updates merge imminent

[Ludovic Courtès]

If there are no objections, we’ll let Hydra build all of ‘core-updates’
and merge it afterwards–i.e., within 2-3 days.

OK?

Ludo’.

Re: core-updates merge imminent

[Ludovic Courtès]

Done!

At the moment most packages are built for x86_64; i686 is a bit behind
and mips64el is a lot behind.

Ludo’.

core-updates merge imminent

[Ludovic Courtès]

Hi Guix!

Hydra is now building all the packages from ‘core-updates’, so unless
something goes wrong, expect it to be merged within 24-48 hours.

We’ll reopen a ‘core-updates’ branch shortly after that.

Ludo’.

Re: core-updates merge imminent

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

 It seems we can soon merge ‘core-updates’.  So far Hydra has been
 building the core packages of that branch, and the only problem left is
 the cross-compilation of the bootstrap GCC (‘%gcc-static’ in
 make-bootstrap.scm), which is annoying but not blocking (see
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59217 for details.)
 Mark has also been building some of it on the Loongson.

I’ve just merged the core-updates branch.  There’s little breakage
AFAICS ;-), but please do check your packages and report anything
suspicious.

Also, the good news is that the cross-compilation issue reported above
is fixed by 0ece707.

Ludo’.