Question about compile packages

[Charles Direg]

 Dear,

How can I modify the flags that any program is compiled with within guix? That
is, I can allow in the gnu-build-system to modify it globally so that I can
add the build flags to any package, for example, add the flags -O2
-march=native -mtune=native so global as I already mentioned, so that these
are added to each package at the time of compilation, this would not be
within the guix development environment, because what I want is that this
compilation is natively for my pc.

As a second question, how could I set the --no-susbtitutes option when
installing the guix system from ISO, since I would like all installed
packages to be compiled natively first?

I really appreciate your kind time and I look forward to your responses.

Sincerely,
~ Abraham Huerta

Utility for copying environments

[raingloom]

Tale as old as time: I was fed up with some software and instead of
patching it I created a workaround. But I think it could be genuinely
useful for some other folks too.
My main use case is gonna be opening new kakoune windows, but it should
be useful more generally as well.

https://git.sr.ht/~raingloom/with-env

Patches and feedback are welcome, as usual.

ps.: please forgive my typos. i really should not have stayed awake to
finish this.

Re: Will 2021 be the year of build systems on gexps?

[Maxim Cournoyer]

Hi Ludo!

Ludovic Courtès  writes:

> Hello!
>
> Ludovic Courtès  skribis:
>
>> Ludovic Courtès  skribis:
>>
>>> Over the last few days I’ve been head-down working on
>>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
>>> systems and packages, so we can say goodbye to
>>> ‘build-expression->derivation’.  And… it’s quite a ride!
>>
>> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
>> it can build ‘guix’ and cross-build things like ‘sed’:
>>
>>   
>> https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>>
>>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>>   currently has unrelated problems)
>
> It’s building and well!
>
>> In terms of performance, there’s still a ~10% slowdown when computing
>> derivations compared to the ‘core-updates’ revision the branch is based
>> on.
>
> I made some improvements yesterday (reducing object cache lookups and
> the number of entries therein), but we’re still in the 10% ballpark.
> WIP branch:
>
> $ git log |head -5
> commit 082df93be3472e0f38970634260af8c432420b35
> Author: Ludovic Courtès 
> Date:   Mon Mar 8 13:59:23 2021 +0100
>
> gnu: docbook-xsl: Move 'use-modules' form to the top level.
> $ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
> /gnu/store/fsrbbi8vfrwwdz2dlyzpfvvnky03nczz-libreoffice-6.4.7.2.drv
> Garbage collection statistics:
>   heap size:87.18 MiB
>   allocated:254.25 MiB
>   GC times: 16
>   time spent in GC: 0.74 seconds (31% of user time)
>
> real  0m2.225s
> user  0m2.415s
> sys   0m0.087s
>
>
> Compared to ‘core-updates’:
>
> $ git log |head -5
> commit b35581bd63d929e83d18f42b067f63efc867353c
> Author: Efraim Flashner 
> Date:   Sun Mar 21 09:42:06 2021 +0200
>
> gnu: openjpeg: Update to 2.4.0.
> $ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
> /gnu/store/irdhm6jx30bgdxvgb0an1mn223rzshkg-libreoffice-6.4.7.2.drv
> Garbage collection statistics:
>   heap size:79.18 MiB
>   allocated:216.51 MiB
>   GC times: 16
>   time spent in GC: 0.74 seconds (33% of user time)
>
> real  0m2.094s
> user  0m2.277s
> sys   0m0.106s

This looks reasonable!

>> Here’s what I’d like to do in the coming days, if that doesn’t interfere
>> with what others have in mind for the upcoming release:
>>
>>   • Monitor build failures due to typos/thinkos made while adjusting
>> build systems;
>>
>>   • Merge on ‘core-updates’.
>
> I’ll go ahead with that if there are no objections.

Sounds good!  Thanks for picking up this work! :-)

Maxim

Re: Release 1.2.1: zstd 1.4.4 -> 1.4.9: grafting or core-updates?

[Léo Le Bouter]

For reference, crossposting:

I pushed 00c67375b17f4a4cfad53399d1918f2e7eba2c7d to core-updates. Your
patch. Thank you for it. Let's watch for upstream zstd fix also.

I pushed 9feef62b73e284e106717a386624d6da90750a3d to master.

Ubuntu released a patch in the mean time, so while we couldnt make such
patch in a timely manner because the backport was non-trivial and
security-sensitive also didnt want to risk failing to fix the flaw
because I don't have much expertise on it, Ubuntu now has done that
work and we can just use it.

Léo


signature.asc
Description: This is a digitally signed message part

Re: GNOME 40 work should be done on Savannah (was: Re: GNOME 40)

[Léo Le Bouter]

Hello!

On Mon, 2021-03-29 at 19:02 -0400, Mark H Weaver wrote:
> This sounds theoretical.  Concretely, what needs do you have that
> aren't
> being met by Savannah?

Per-branch access control

> I don't understand this.  It seems to me the opposite.
> 
> If I want to contribute to this external 'wip' branch, I need to
> arrange
> for access.  Ditto for any other Guix committer who wants to work on
> it.
> That's added "bureaucracy" entailed by your approach that would not
> be
> needed for 'wip' branches on Savannah.

Cbaines is more responsive and has much lower requirements than what
the "Commit Access" for GNU Guix itself requires. It's as if we created
a third party git repo for both of us Raghav and myself then
collaborated there except through Cbaines's infra we get CI
infrastructure for free.

> On the other hand, maybe your point is that you'd like to allow
> direct
> commit access to this 'wip' branch by people who don't have commit
> access to Savannah.  If that's the goal, I find that objectionable,
> because when this branch is finally merged, all of those commits will
> suddenly get dumped into Savannah.  That increases "risk" from my
> perspective.
> 
> I actively do not want commits getting into Savannah without an
> existing
> Guix committer taking responsibility for them.  Your approach
> effectively creates a loophole for non-committers to potentially
> introduce many commits into the official Guix repository in a way
> that
> is likely to not get adequate oversight.

Why would it not get adequate oversight? It's just an easier way to
collaborate on patches, but the patchset would be sent over to guix-
patches before getting merged to master or else.

In general I don't agree with such gatekeeping of access to wip
branches because it actively hinders the development of GNU Guix by
non-committers, and many non-committers would like to get involved more
but they are barred by the commit access requirement.

> * * *
> 
> I'd strongly prefer for this work to be done on Savannah.  If this
> were
> a fringe branch of marginal interest, it might make sense to have it
> elsewhere, but this is core Guix desktop work that's likely to be of
> interest to a large segment (plausibly a majority) of our community.
> IMO, it belongs in our official git repository.
> 
> Thoughts?
> 
>   Mark

The people that work on it now are Raghav and me, and Raghav does not
have commit access yet, so that's the only way we can work and
cooperate now. We don't have a choice. If and when Raghav's commit
access application is approved then we can move to Savannah.

I don't feel like people should be barred to contribute to that GNOME
40 upgrade because they arent an approved committer. That doesnt feel
inclusive to me.

If you want to work on this GNOME upgrade however, that help is more
than welcome, in this particular situation probably we can work on
getting Raghav's commit access application approved then your concerns
will be sorted out as no other non-committer participant seemed to show
up.

Léo


signature.asc
Description: This is a digitally signed message part

GNOME 40 work should be done on Savannah (was: Re: GNOME 40)

[Mark H Weaver]

Christopher Baines  writes:

> Mark H Weaver  writes:
>> How is it more flexible than a "wip-*" branch on Savannah?
>
> I wouldn't use quite the same words as Léo, but from my perspective,
> controlling access to particular branches (master, staging,
> core-updates, ...) on Savannah is a good thing, as it reduces risk.

I don't see much risk here.  You're talking about a 'wip' branch that
almost no one will be using anyway.  We already trust all Guix
committers with our master branch, which directly and immediately
affects any Guix user who updates their system at the right time.

If someone commits something inappropriate to a 'wip' branch, we can all
easily see that they did so, investigate more closely, and optionally
revert the changes.

Léo Le Bouter  writes:

> On Sun, 2021-03-28 at 16:48 -0400, Mark H Weaver wrote:
>> How is it more flexible than a "wip-*" branch on Savannah?
>
> Because as the GNU Guix project we have no control on the forge to
> catter it to our own needs,

This sounds theoretical.  Concretely, what needs do you have that aren't
being met by Savannah?

> because there is bureaucracy involved with approving new committers so
> they can work on wip branches (shouldnt be necessary).

I don't understand this.  It seems to me the opposite.

If I want to contribute to this external 'wip' branch, I need to arrange
for access.  Ditto for any other Guix committer who wants to work on it.
That's added "bureaucracy" entailed by your approach that would not be
needed for 'wip' branches on Savannah.

On the other hand, maybe your point is that you'd like to allow direct
commit access to this 'wip' branch by people who don't have commit
access to Savannah.  If that's the goal, I find that objectionable,
because when this branch is finally merged, all of those commits will
suddenly get dumped into Savannah.  That increases "risk" from my
perspective.

I actively do not want commits getting into Savannah without an existing
Guix committer taking responsibility for them.  Your approach
effectively creates a loophole for non-committers to potentially
introduce many commits into the official Guix repository in a way that
is likely to not get adequate oversight.

* * *

I'd strongly prefer for this work to be done on Savannah.  If this were
a fringe branch of marginal interest, it might make sense to have it
elsewhere, but this is core Guix desktop work that's likely to be of
interest to a large segment (plausibly a majority) of our community.
IMO, it belongs in our official git repository.

Thoughts?

  Mark

Re: GNOME 40

[Léo Le Bouter]

On Sun, 2021-03-28 at 16:48 -0400, Mark H Weaver wrote:
> How is it more flexible than a "wip-*" branch on Savannah?
> 
>  Thanks,
>Mark

Because as the GNU Guix project we have no control on the forge to
catter it to our own needs, because there is bureaucracy involved with
approving new committers so they can work on wip branches (shouldnt be
necessary).


signature.asc
Description: This is a digitally signed message part

Re: [PATCHES] ImageMagick security updates without grafting

[Mark H Weaver]

Hi Maxime,

Maxime Devos  writes:

> On Sun, 2021-03-28 at 17:37 -0400, Mark H Weaver wrote:
>> One thing to be very careful about is to only use 'gtk-doc/stable',
>> 'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
>> moreover to make sure that no references to these */stable packages
>> remain in any package outputs.
>> 
>> Of course, if any package retains references to its 'native-inputs',
>> that's always a bug, but I wouldn't be surprised if such bugs exist in
>> Guix.  Such bugs might be relatively harmless now (except when
>> cross-compiling), but they could become a security bug if a package
>> retains a reference to 'imagemagick/stable'.
>
> I'll be careful!
>
>> On my own system and user profile, which includes GNOME, I'm glad to
>> report that I have *no* references to 'imagemagick' at all, not even to
>> its newest release, and that's my strong preference.
>
> Note to self, before I forget how to test this:
>
> guix build $PACKAGES
> # maybe guix build $PACKAGES --no-grafts?
> guix graph --type=references $PACKAGES
> # ^ look in output for "imagemagick".

For the record, it seems that this command gives false positives.  As
pointed out in , the output of that command
suggests that 'inkscape' retains references to 'imagemagick', but that
turns out to be false, at least on my system.

I suppose the behavior of "guix graph" here makes sense, and is likely
_not_ a bug, because IIUC "guix graph" does its work without requiring
'imagemagick' to be built, and therefore it cannot know whether
imagemagick's build system would retain a reference to a native-input
during its build process.  IMO, it would be inappropriate for "guix
graph" to *assume* that references to native-inputs will not retained.

The tool I expect to be reliable here is "guix gc -R".  For example, I
check for references to 'imagemagick' in my system and user profiles
with the following commands:

--8<---cut here---start->8---
mhw@jojen ~$ guix gc -R $(readlink /run/current-system) | grep -i imagemagick
mhw@jojen ~$ guix gc -R $(readlink -f ~/.guix-profile) | grep -i imagemagick
--8<---cut here---end--->8---

 Thanks,
   Mark

Re: Needed: tooling to detect references to buggy */stable packages (was: Re: [PATCHES] ImageMagick security updates without grafting)

[Ricardo Wurmus]

Mark H Weaver  writes:

> Earlier, I wrote:
>> One thing to be very careful about is to only use 'gtk-doc/stable',
>> 'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
>> moreover to make sure that no references to these */stable packages
>> remain in any package outputs.
>>
>> Of course, if any package retains references to its 'native-inputs',
>> that's always a bug, but I wouldn't be surprised if such bugs exist in
>> Guix.  Such bugs might be relatively harmless now (except when
>> cross-compiling), but they could become a security bug if a package
>> retains a reference to 'imagemagick/stable'.
>
> It occurs to me that we will need some tooling to ensure that no
> references to these buggy "*/stable" packages end up in package outputs
> that users actually use.  Otherwise, it is likely that sooner or later,
> a runtime reference to one of these buggy packages will sneak in to our
> systems.

The gnu-build-system takes a keyword #:disallowed-references that could
be used here.

-- 
Ricardo

Re: GNOME 40

[Raghav Gururajan]

Hello Guix!


If anyone is curious of the work or wants to participate, we are
working there:
https://git.guix-patches.cbaines.net/guix-patches/log/?h=wip-gnome-40

The branch is based on core-updates and we will rebase it every now and
then, as well as merging patches to official core-updates as we feel it
is necessary.

cbaines gave us access to the git repo, it's not official so it's more
flexible, right now the access cbaines gives is all in and trustful,
but in the future they plan on getting branch access control with
https://gitolite.com/gitolite/ so that people can be given access to
wip branches with associated continuous integration infrastructure for
development.

If anyone wants to participate, please reach to cbaines about access.


Now we have a chart for navigation. \o/

Chart: https://calc.disroot.org/guix-gnome.html

Regards,
RG.



OpenPGP_signature
Description: OpenPGP digital signature

Re: GNOME 40

[Christopher Baines]

Mark H Weaver  writes:

> Léo Le Bouter  writes:
>
>> If anyone is curious of the work or wants to participate, we are
>> working there:
>> https://git.guix-patches.cbaines.net/guix-patches/log/?h=wip-gnome-40
>>
>> The branch is based on core-updates and we will rebase it every now and
>> then, as well as merging patches to official core-updates as we feel it
>> is necessary.
>>
>> cbaines gave us access to the git repo, it's not official so it's more
>> flexible,
>
> How is it more flexible than a "wip-*" branch on Savannah?

I wouldn't use quite the same words as Léo, but from my perspective,
controlling access to particular branches (master, staging,
core-updates, ...) on Savannah is a good thing, as it reduces risk. I
don't think that's relevant for all branches though, it would be better
if more people could collaborate through wip-* branches (if that's
useful to them).

As Savannah doesn't allow for access control per branch (or even per
project repository), currently to give someone access to anything, you
have to give them access to everything. Maybe that's possible to
improve, but using some other Git repository hosted elsewhere is one
workaround.


signature.asc
Description: PGP signature

Re: armhf-linux substitutes

[Mathieu Othacehe]

Hello,

> Mathieu, what’s preventing us from doing armhf-linux builds again?  We
> could use the OverDrives for that (with an upper bound though), along
> with the SBCs in machines-for-berlin.scm.
>
> That won’t be enough to keep up, so perhaps we’ll have to restrict
> armhf-linux builds to the “core” subset or the release-manifest.scm.

With emulating builds performing badly, I'm considering building only the
'core subset for armhf and aarch64 and only on native hardware. That
would require to have the other overdrives up & running I guess.

Thanks,

Mathieu

Re: Needed: tooling to detect references to buggy */stable packages (was: Re: [PATCHES] ImageMagick security updates without grafting)

[Maxime Devos]

On Sun, 2021-03-28 at 18:33 -0400, Mark H Weaver wrote:
> Earlier, I wrote:
> > One thing to be very careful about is to only use 'gtk-doc/stable',
> > 'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
> > moreover to make sure that no references to these */stable packages
> > remain in any package outputs.
> > 
> > Of course, if any package retains references to its 'native-inputs',
> > that's always a bug, but I wouldn't be surprised if such bugs exist in
> > Guix.  Such bugs might be relatively harmless now (except when
> > cross-compiling), but they could become a security bug if a package
> > retains a reference to 'imagemagick/stable'.

It just occurred to me: could we automatically add all native-inputs
to #:disallowed-references when cross-compiling?  This shouldn't break
any packages, except possibly when cross-compiling.

Or stronger, add all native-inputs to #:disallowed-references (unless
they are also in inputs or propagated-inputs), even when compiling
natively?

Problems include:
* I guess a world rebuild, as the derivations would be different.
* In some places we have the following pattern:

(native-inputs
 `(("autoconf" ,autoconf)
   ("automake" ,automake)
   ("pkg-config" ,pkg-config)
   ,@(if (%current-target-system)
 `(("guile" ,guile-3.0))   ;for 'guild compile' and 'guile-3.0.pc'
 '(
(inputs
 `(("guile" ,guile-3.0)
   ("lzlib" ,lzlib)))
(synopsis "Guile bindings to lzlib")

  The (if (%current-target-system) ...) would need to be made unconditional.
* I guess an option to disable this behaviour might be useful.

> It occurs to me that we will need some tooling to ensure that no
> references to these buggy "*/stable" packages end up in package outputs
> that users actually use.  Otherwise, it is likely that sooner or later,
> a runtime reference to one of these buggy packages will sneak in to our
> systems.
> 
> An initial idea is that these "*/stable" packages could have a package
> property (perhaps named something like 'build-time-only') that indicates
> that references to its outputs should not occur within the outputs of
> any other package that does not have that same property.

Would this be (a) something enforced by the build process (using
#:disallowed-references or #:allowed-references), or (b) a linter?

> We'd also need to somehow ensure that users don't install these
> 'build-time-only' packages directly, at least not without an additional
> option (e.g. --force-unsafe-build-time-only) to override it.

What about a developer running "guix environment eom"?  IIUC, this would
make the developer vulnerable (at least, once I've gotten around replacing
the 'gtk-doc' input with 'gtk-doc/stable'), so it might make sense to
replace /stable -> unstable packages here.

However, now the developer ends up with a different set of packages than
wil be seen in the build environment ...

> Additionally, it might be good to issue warnings if 'build-time-only'
> packages are not hidden,

This seems good to me.  This should prevent
"guix install imagemagick@bad-version".

>  or if they are found within the 'inputs' or
> 'propagated-inputs' fields of any package that's not also
> 'build-time-only'.  Both of these last two checks have loopholes,
> however, so they are not reliable indicators.

But these (automatic "guix lint") checks could still catch many
problems in practice before they are committed!  

> Thoughts?  Other proposals?

Is this something you will be writing "guix lint" checkers (or other
checkers) for yourself?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part