Re: Proposal: Build timers

[Jacob Hrbek]

I think you are overcomplicating the implementation.. What I am proposing is to 
store the time value before and after the build and then log the subtraction of 
these two values per package (or even per package's phase).

For storage it can be either/both:
1. locally: Storing the value somewhere on the system and adding up to it each 
build to provide more accurate average.

**optionally** This local database can be shared across multiple systems that 
add values to it like simple listener waiting for POST requests.

- within the guix repo: Since we are already building the package we can take 
the time and then do the provided math in reverse to calculate the time:

Build took 100 sec on system with 8 threads at 2.4 Ghz max cpu frequency:

100 * (2.4 * 8) = 1920 (build time value per one thread at 1 Ghz)

Building the package on system with 2 threads at 2.4 Ghz max cpu frequency:

1920 / (2 * 2.4) = 400

We can then assume that the build will take 1920/400=4.8 -> 4.8 times 
longer on this system.

The math might need to be adjusted, but it seems to be sufficiently accurate 
through my testing, fwiw I see that `(max cpu frequency * cpu threads)` is an 
important component of the equasion using the analogy of a (possibly silly) 
"pokemon battle" assuming interpreting a mathematical equasion to define the 
Health Points of the package and damage per second of the CPU then simply 
substracting these two values to determine how long it will take to build alike 
package has 500 HP -> Needs a CPU that deals 100 HP to complete in 5 sec or CPU 
that deals 50 HP to finish in 10 sec.

About accuracy: I highly doubt that we need to worry about the system noise as 
that will be mitigated after enough systems submit average build time with 
their max CPU frequency and threads used.. we shouldn't really bother past that 
at the current stage and we can always add additional metadata if needed.

Either way if we decide to not implement the logging of the build time I would 
still argue that we should still consider just plain outputting of the time 
needed to complete the phase and package as I see it as a best practice for 
monitoring and assuming that it's very cheap (5~20 cpu cycles?) in terms of 
processing resources to do so.

-- Jacob "Kreyren" Hrbek

Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐

On Tuesday, November 23rd, 2021 at 1:06 AM, zimoun  
wrote:

> Hi,
> 

> On Mon, 22 Nov 2021 at 22:02, Jacob Hrbek krey...@rixotstudio.cz wrote:
> 

> > See the proposal in https://git.dotya.ml/guix.next/GUIX.next/issues/5
> 

> If I understand well your proposal, you are suggesting to attach a value
> 

> ’build time’. While I understand it could be useful for monitoring;
> 

> especially CI (Cuirass or Build Coordinator) – but it appears useless
> 

> for your use case. Where do you want to attach such value? I think it
> 

> is not affordable to add another field (or ’properties’) for all the
> 

> packages.
> 

> When discussing the Cuirass revamp, it had been mentioned to grab the
> 

> Cuirass database and then try some analytics to infer heuristics helping
> 

> for a better scheduling strategy. However, the task is not as easy as
> 

> it appears at first. Some builds are blocked by unrelated IO
> 

> operations, e.g., [1], thus it is hard to distinguish between a
> 

> regression or something unexpected in the build farms. Something useful
> 

> for monitoring, but hard to exploit for local builds, IMHO.
> 

> Other said, the “telemetry“ you are suggesting require non-trivial
> 

> filtering to gain the robust feature you expect, again IMHO.
> 

> Last, build-time depends on the environment (how the machine is
> 

> stressed), and for instance, I do not want to stop a build because on
> 

> average people are building it using X time when my machine builds it
> 

> today using X+y time (because it is CPU stressed by something else or
> 

> whatever). Well, I am doubtful that the standard error would not be too
> 

> much compared to the mean; other said, my guess is a flat Gaussian
> 

> function because of heterogeneous hardware and various levels of stress
> 

> for the same build.
> 

> To be explicit, I do not think Guix should take care of this. From my
> 

> opinion, if the build farm does not have the substitute (guix weather),
> 

> it is a bad sign the package will build locally; therefore, if resources
> 

> are limited, before building locally, I would inspect the output of
> 

> build farms (ci.guix.gnu.org and bordeaux.guix.gnu.org). Obviously, it
> 

> depends your target architecture and some are poorly supported, sadly.
> 

> 1: http://issues.guix.gnu.org/issue/51787
> 

> Cheers,
> 

> simon

publickey - kreyren@rixotstudio.cz - 0x1677DB82.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: Proposal: Build timers

[zimoun]

Hi,

On Mon, 22 Nov 2021 at 22:02, Jacob Hrbek  wrote:

> See the proposal in https://git.dotya.ml/guix.next/GUIX.next/issues/5

If I understand well your proposal, you are suggesting to attach a value
’build time’.  While I understand it could be useful for monitoring;
especially CI (Cuirass or Build Coordinator) – but it appears useless
for your use case.  Where do you want to attach such value?  I think it
is not affordable to add another field (or ’properties’) for all the
packages.

When discussing the Cuirass revamp, it had been mentioned to grab the
Cuirass database and then try some analytics to infer heuristics helping
for a better scheduling strategy.  However, the task is not as easy as
it appears at first.  Some builds are blocked by unrelated IO
operations, e.g., [1], thus it is hard to distinguish between a
regression or something unexpected in the build farms.  Something useful
for monitoring, but hard to exploit for local builds, IMHO.

Other said, the “telemetry“ you are suggesting require non-trivial
filtering to gain the robust feature you expect, again IMHO.

Last, build-time depends on the environment (how the machine is
stressed), and for instance, I do not want to stop a build because on
average people are building it using X time when my machine builds it
today using X+y time (because it is CPU stressed by something else or
whatever).  Well, I am doubtful that the standard error would not be too
much compared to the mean; other said, my guess is a flat Gaussian
function because of heterogeneous hardware and various levels of stress
for the same build.

To be explicit, I do not think Guix should take care of this.  From my
opinion, if the build farm does not have the substitute (guix weather),
it is a bad sign the package will build locally; therefore, if resources
are limited, before building locally, I would inspect the output of
build farms (ci.guix.gnu.org and bordeaux.guix.gnu.org).  Obviously, it
depends your target architecture and some are poorly supported, sadly.


1: 


Cheers,
simon

Proposal: Build timers

[Jacob Hrbek]

See the proposal in https://git.dotya.ml/guix.next/GUIX.next/issues/5

-- Jacob "Kreyren" Hrbek

Sent with ProtonMail Secure Email.

publickey - kreyren@rixotstudio.cz - 0x1677DB82.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: Replacing polkit by polkit-duktape on core-updates-frozen ?

[Maxim Cournoyer]

Hello again,

I've now tried this:

--8<---cut here---start->8---
gnu: polkit: Define polkit package variable based on architecture.

* gnu/packages/polkit.scm (polkit): Rename to...
(polkit*): ... this.
(polkit-duktape): Adjust to inherit from polkit*.
(polkit-for-system): New procedure.
(polkit): New variable.

1 file changed, 44 insertions(+), 27 deletions(-)
gnu/packages/polkit.scm | 71 
---

modified   gnu/packages/polkit.scm
@@ -48,7 +48,7 @@ (define-module (gnu packages polkit)
   #:use-module (gnu packages qt)
   #:use-module (gnu packages xml))
 
-(define-public polkit
+(define-public polkit*
   (package
 (name "polkit")
 (version "0.120")
@@ -151,32 +151,49 @@ (define-public polkit
 ;;; Variant of polkit built with Duktape, a lighter JavaScript engine compared
 ;;; to mozjs.
 (define-public polkit-duktape
-  (package/inherit polkit
-(name "polkit-duktape")
-(source
- (origin
-   (inherit (package-source polkit))
-   (patches
-(append
-(search-patches "polkit-use-duktape.patch")
-(origin-patches (package-source polkit))
-(arguments
- (substitute-keyword-arguments (package-arguments polkit)
-   ((#:configure-flags flags)
-`(cons "--with-duktape" ,flags))
-   ((#:phases phases)
-`(modify-phases ,phases
-   (add-after 'unpack 'force-gnu-build-system-bootstrap
- (lambda _
-   (delete-file "configure")))
-(native-inputs
- (append `(("autoconf" ,autoconf)
-   ("automake" ,automake)
-   ("libtool" ,libtool)
-   ("pkg-config" ,pkg-config))
- (package-native-inputs polkit)))
-(inputs (alist-replace "mozjs" `(,duktape)
-   (package-inputs polkit)
+  (let ((base polkit*))
+(package/inherit base
+  (name "polkit-duktape")
+  (source
+   (origin
+ (inherit (package-source base))
+ (patches
+  (append
+  (search-patches "polkit-use-duktape.patch")
+  (origin-patches (package-source base))
+  (arguments
+   (substitute-keyword-arguments (package-arguments base)
+ ((#:configure-flags flags)
+  `(cons "--with-duktape" ,flags))
+ ((#:phases phases)
+  `(modify-phases ,phases
+ (add-after 'unpack 'force-gnu-build-system-bootstrap
+   (lambda _
+ (delete-file "configure")))
+  (native-inputs
+   (append `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("libtool" ,libtool)
+ ("pkg-config" ,pkg-config))
+   (package-native-inputs base)))
+  (inputs (alist-replace "mozjs" `(,duktape)
+ (package-inputs base))
+
+
+(define (polkit-for-system system)
+  "Return a polkit package that can be built for SYSTEM; that is, either the
+regular polkit that requires mozjs or its duktape variant."
+  (if (string-prefix? "x86_64" system)
+  polkit*
+  polkit-duktape))
+
+;;; Define a top level polkit variable that can be built on any of the
+;;; supported platforms.  This is to work around the fact that our
+;;; mrustc-bootstrapped rust toolchain currently only supports the x86_64
+;;; architecture.
+(define-syntax polkit
+  (identifier-syntax (polkit-for-system
+  (or (%current-system) (%current-target-system)
 
 (define-public polkit-qt
   (package
--8<---cut here---end--->8---

But my "test" fails the same:

--8<---cut here---start->8---
$ ./pre-inst-env guix build --system=aarch64-linux \
-e '(@ (gnu packages polkit) polkit)' -n |& grep polkit
/gnu/store/dw11y85xfsb8hcg7w2cw57f1xfs4i74m-polkit-0.120.drv
/gnu/store/ric7yf4ra2p14p29fwsh18m1nakciakv-polkit-0.120.tar.xz
--8<---cut here---end--->8---

I'd have expected to see 'polkit-duktape' as the name of the package in
the output.

Thanks,

Maxim

Re: Replacing polkit by polkit-duktape on core-updates-frozen ?

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hi Maxim,
>
> Maxim Cournoyer  skribis:
>
>> +;;; Define a top level polkit variable that can be built on any of the
>> +;;; supported platforms.  This is to work around the fact that our
>> +;;; mrustc-bootstrapped rust toolchain currently only supports the x86_64
>> +;;; architecture...
>> +(define-public polkit
>> +  (if (string-prefix? "x86_64"
>> +  (or (%current-system) (%current-target-system)))
>> +  polkit*
>> +  polkit-duktape))
>
> This would instead have to be a macro, similar to the ‘pkg-config’
> macro, so that (%current-system) is checked at the right time rather
> than when loading this module.

Thanks for tipping in!

> However, since the only different between polkit and polkit-duktape
> (IIUC) is that an extra patch is applied for the latter, I would instead
> suggest adding a conditional build phase that applies the patch on
> non-x86_64 systems.

I'd still rather keep the distinct package approach, so that x86* users
may still find about polkit-duktape and use it instead of polkit (via
input rewriting) if they so wish.

Thanks,

Maxim

Re: default tar format for "make dist" and patch file length

[Maxim Cournoyer]

Hi Ludo,

Ludovic Courtès  writes:

[...]

> Any objections against the patch below?
>
> diff --git a/configure.ac b/configure.ac
> index 84592f6041..341cff8fbd 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -8,7 +8,7 @@ AC_INIT([GNU Guix],
>[https://www.gnu.org/software/guix/])
>  AC_CONFIG_AUX_DIR([build-aux])
>  
> -AM_INIT_AUTOMAKE([1.14 gnu silent-rules subdir-objects \
> +AM_INIT_AUTOMAKE([1.14 gnu tar-ustar silent-rules subdir-objects \
>   color-tests parallel-tests -Woverride -Wno-portability])
>  
>  # Enable silent rules by default.
>
>
> I tried it and there’s no tar warning and the file looks just fine,
> including the long file names:
>
> $ make dist -j5
> […]
> make[2]: Leaving directory '/home/ludo/src/guix'
> tardir=guix-1.3.0.8612-7cad44-dirty && tar --format=ustar -chf - "$tardir" | 
> eval GZIP= gzip --best -c >guix-1.3.0.8612-7cad44-dirty.tar.gz
> make[1]: Leaving directory '/home/ludo/src/guix'
> if test -d "guix-1.3.0.8612-7cad44-dirty"; then find 
> "guix-1.3.0.8612-7cad44-dirty" -type d ! -perm -200 -exec chmod u+w {} ';' && 
> rm -rf "guix-1.3.0.8612-7cad44-dirty" || { sleep 5 && rm -rf 
> "guix-1.3.0.8612-7cad44-dirty"; }; else :; fi
> $ tar tvf guix-1.3.0.8612-7cad44-dirty.tar.gz |grep patches/xplanet
> -rw-r--r-- ludo/users 1252 2020-04-06 00:14 
> guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-libimage_gif.c.patch
> -rw-r--r-- ludo/users  765 2020-04-06 00:14 
> guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-xpUtil-Add2017LeapSecond.cpp.patch
> -rw-r--r-- ludo/users 6990 2020-04-06 00:14 
> guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-cxx11-eof.patch
> -rw-r--r-- ludo/users  783 2020-04-06 00:14 
> guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-libdisplay_DisplayOutput.cpp.patch
>
>> Seems a good thing to be as portable as can be, especially since 200
>> chars patch file names wouldn't look good in the sources anyway ;-).
>
> I don’t think we care about “portability” as understood in the context
> of Automake, which roughly translates to “support SunOS 2 and HP-UX’s
> 1991 ‘tar’ implementation”.  :-)

Hehe.

The patch LGTM.

Thanks!

Maxim

Re: Thursday 18th: ‘core-updates-frozen’ sprint!

[Maxim Cournoyer]

Hello,

zimoun  writes:

> Hi Ludo,
>
> On Mon, 22 Nov 2021 at 14:21, Ludovic Courtès  wrote:
>
>> >> … there’s probably more.
>
> Julia is annoying,  Because the test suite sometimes passes, sometimes
> not.  Well, I am not even sure it is the same part that fails.  I
> notice that "guix build julia --without-tests=julia" then "guix build
> julia" tends to works. :-)
>
> Any idea to debug this?

No idea how to debug this, but when faced with nondeterministic tests,
here's how I proceed:

1. Report them upstream.
2. Disable the problematic tests.  You may need to disable a whole test suite of
similarly faulted tests, if they all behave the same.

Thanks,

Maxim

Re: [core-updates-frozen] Different variants of Python packages in the same profile?

[Ricardo Wurmus]

Ludovic Courtès  writes:


Hi,

Ricardo Wurmus  skribis:


on the core-updates-frozen branch we have a minor problem:
python-numpy has been upgraded to 1.21, but python-numba can 
only 
be built with 1.20[1].  I added python-numpy-1.20 and made

python-numba use it.


How big of an effort would it be to modify numba so that it 
works with

1.21?  With luck, that effort has already been made upstream?


I spoke too soon: 
https://github.com/numba/numba/issues/7175#issuecomment-975470113


 “Numba 0.55 release candidates will be out within the next week 
 or so (hopefully!), these will have NumPy 1.21.x series 
 support.”


So soon after the merge we will be able to switch back to numpy 
1.21 as the default.


--
Ricardo

Re: [core-updates-frozen] Different variants of Python packages in the same profile?

[Ricardo Wurmus]

Hi,

Ludovic Courtès  writes:


Ricardo Wurmus  skribis:


on the core-updates-frozen branch we have a minor problem:
python-numpy has been upgraded to 1.21, but python-numba can 
only 
be built with 1.20[1].  I added python-numpy-1.20 and made

python-numba use it.


How big of an effort would it be to modify numba so that it 
works with

1.21?  With luck, that effort has already been made upstream?


It hasn’t.  Upstream doesn’t seem to know exactly how to 
accomplish this.  There’s a breaking change in numpy (something 
relating to typing of ufuncs) and it leads to incorrect type 
errors in numba AFAIU.  This is not something I’d feel comfortable 
just hacking around.


--
Ricardo

Re: Using G-Expressions for public keys (substitutes and possibly more)

[Liliana Marie Prikler]

Am Montag, den 22.11.2021, 14:27 +0100 schrieb Ludovic Courtès:
> Hi,
> 
> Liliana Marie Prikler  skribis:
> 
> > I think we would probably want to improve on this end in the guile-
> > gcrypt module, i.e. have a public-key "constructor" that returns a
> > canonical-sexp and so on.  WDYT?
> 
> I don’t find it very compelling given there’s already
> ‘sexp->canonical-sexp’ & co.  WDYT?
sexp->canonical-sexp is from the comment surrounding it a rather hacky
beast.  I think (guile-)gcrypt could very well benefit from having a
better sexp construction API, but again that's outside the scope of
Guix.

But since you're asking, I do feel there's a lot guile-gcrypt could
borrow from Guix, just not right now in an efficient manner thanks to
needing to jump through the sexp->canonical-sexp hoops.  It could have
semantic constructors and field sanitization à la (guix records) for
example.  WDYT?

Re: ZFS part of Guix? RFC? (Re: Effectively force all GNOME users to locally compile ZFS?)

[pelzflorian (Florian Pelz)]

On Sun, Nov 21, 2021 at 11:54:15AM +0100, pelzflorian (Florian Pelz) wrote:
> raid5atemyhomework wrote patches to add ZFS to Guix
> .  I put them in CC.  That there is
> no decision on ZFS and their patches is bad.  Maybe their patches
> would be for the RFC model to decide?

Maybe there is consensus that adding ZFS is a legal risk.

There is no consensus on the amount of risk.

Are there two camps,

- one unwilling to impose any such risk on users (i.e. close as
  won’t-fix  and remove traces of
  ZFS)

- one “Worse is better” camp that likes to move fast and break things?
  I think of
  ,
  even though Katherine Cox-Buday did not get involved here at all.

Then who decides?

I don’t actually care about ZFS myself, but there should be a decision
because I think current badly supported ZFS makes people here unhappy.

Regards,
Florian

Re: ZFS part of Guix? RFC? (Re: Effectively force all GNOME users to locally compile ZFS?)

[Denis 'GNUtoo' Carikli]

On Sun, 21 Nov 2021 11:54:15 +0100
"pelzflorian (Florian Pelz)"  wrote:

> Hello Denis.  Thank you for your write-up.
Hi,

> raid5atemyhomework wrote patches to add ZFS to Guix
> .  I put them in CC.  That there is
> no decision on ZFS and their patches is bad.  Maybe their patches
> would be for the RFC model to decide?
> 
> As for Denis’ arguments
> :
> 
> IANAL but I would dispute this:
> 
> On Sun, Nov 21, 2021 at 02:33:24AM +0100, Denis 'GNUtoo' Carikli
> wrote:
> > If I take individual drivers from Linux which are under the GPLv2 or
> > the GPLv2 or later, and I combine them in a new combined work on a
> > new git repository with code under the CDDL license, this is not
> > legal either. And here too I hope that there is some consensus on
> > that too.
> 
> If we are talking about source code, the GPLv2 files and the CDDL
> files are not intertwined into a combined work at all.  They are just
> in the same git repo on the same file-system.
Just adding GPLv2 and CDDL files in the same repository should not be a
problem as far as I know. 

For instance you can have a kernel module under the GPLv2, and a
userspace tool under the CDDL license that doesn't use code from the
GPLv2 driver, and in that case both don't constitute a combined work.

If the userspace tool uses syscalls from the kernel, the kernel headers
license has an exception for that so it shound't be an issue either.

However I've reviewed a bit the ZFS kernel driver and some files are
under the CDDL license, and that driver also uses functions from the
Linux kernel, and it needs to be directly linked to Linux to uses these
functions.

I didn't take into account the fact that the ZFS driver also has
GPLv2(+?) code or the structure of the repository because having
files under the CDDL licenses that are compiled in the ZFS driver and
having that driver use Linux function (through linking) is sufficient
to make sure that this is a combined work of Linux and that driver.

And since Linux's GPLv2[1] and the CDDL[2] are incompatible, even in
source code form the ZFS driver is not legal as it violates Linux's
GPLv2 license.

Adding extra layer(s) of indirection with code under GPLv2 compatible
licenses won't change anything here as the ZFS driver links with Linux
anyway and it uses functions in Linux.

As for the GPLv2 in the ZFS driver, in the module directory of
zfs-2.1.1, we have several files under the GPLv2 license or compatible
licenses.

If I "grep -i gpl" in the module directory, it gives several files, but
all the files I found are OK with that are OK with regard with the
GPLv2:
- lua/lapi.c has 'ZFS_MODULE_LICENSE("Dual MIT/GPL")', so we can can
  probably assume that GPL is the GPLv2 and use it under the GPLv2 here.
- The following files are under the GPLv2 or BSD 2 clauses, here we can
  use them under the GPLv2, so it's OK:
  - zcommon/zfs_fletcher_aarch64_neon.c
  - zcommon/zfs_fletcher_intel.c
  - zcommon/zfs_fletcher_sse.c
  - zcommon/zfs_fletcher_superscalar4.c: 
  - zcommon/zfs_fletcher_superscalar.c
- Finally, zstd/zfs_zstd.c is under the BSD 3 clauses, and also has
  "ZFS_MODULE_LICENSE("Dual BSD/GPL");" inside. In any case that BSD
  license isn't incompatible with the GPL, and we can use the GPL in
  "Dual BSD/GPL", so we're good in either cases.

As for the problematic symbols, for instance dequeue_signal is exported
by Linux and it's used by the ZFS driver.

To find about that you can use the following command in a Linux git
checkout to find the list of exported symbols:
> git grep -P "EXPORT_SYMBOL(_GPL)?\(.*\);"

And then the idea is to grep for them in the module directory, and
check if they are reimplemented by the ZFS module or not.

Another way to do that check would be to look at the module (the .ko
file) with nm or a similar tool and look at the undefined symbols (U).

As I understand from what Bradley Kuhn told me, the EXPORT_SYMBOL_GPL
macro name is misleading and It doesn't mean that one can use symbols
exported by EXPORT_SYMBOLS without having to abide by the GPL, and I
need to look at EXPORT_SYMBOLS_GPL history to understand that in more
details.

But we don't need to do that either here as the dequeue_signal is
exported with EXPORT_SYMBOL_GPL anyway (it's in kernel/signal.c Linux)
and AFAIK it's not reimplemented by the ZFS driver either.

And I didn't check how many symbols from Linux are used but one is
enough to be an issue.

References:
---
[1] Many files that are being compiled in Linux are under the GPLv2
only or compatible licenses.
[2] The ZFS driver has code under the CDDL license that is compiled in
the ZFS driver.

Denis.


pgp3zoRSiktvQ.pgp
Description: OpenPGP digital signature

Re: Flag day for simplified package inputs

[Jelle Licht]

Hello there,

Ludovic Courtès  writes:

> Hi,
>
> Jelle Licht  skribis:
>
>> When applying this and future bulk changes, could we perhaps list the
>> specific commits (+ commented shortlog line) in a file. To clarify, if
>> we were to store these commits in `.git-blame-ignore-revs', later on we
>> can instruct users to run:
>>
>> git config blame.ignoreRevsFile .git-blame-ignore-revs
>>
>> to ignore the bulk change for git blame purposes.
>
> Sounds like a good idea.  There’s no particular convention regarding the
> file name of ‘.git-blame-ignore-revs’, right?

As far as I know there is no established convention, although after
searching around a bit I found that Sarah Morgensen proposed the exact
same thing in September. It seems the chromium project also follows the
same naming scheme, so we won't be totally idiosyncratic at least :).

Re: Using G-Expressions for public keys (substitutes and possibly more)

[Jelle Licht]

Ludovic Courtès  writes:

> Hi,
>
> Liliana Marie Prikler  skribis:
>
>> I think we would probably want to improve on this end in the guile-
>> gcrypt module, i.e. have a public-key "constructor" that returns a
>> canonical-sexp and so on.  WDYT?
>
> I don’t find it very compelling given there’s already
> ‘sexp->canonical-sexp’ & co.  WDYT?

Well, the issue here is 'knowing' what sexp to pass along to that
function in the first place. Are Liliana & I missing something
obvious here?

I had to take a string representation of a valid canonical-sexp, and
pass it through string->canonical-sexp and canonical-sexp->sexp. It's
definitely not an issue for managing my local configuration, but it
seems silly to force _anyone_ wanting to write a canonical-sexp as a
sexp through this REPL adventure, for each kind of canonical-sexp.

- Jelle

Help with package AppImage support

[Ekaitz Zarraga]

Hi,

I'm trying to package some AppImage support for later adding it to `guix pack`.

I having issues with AppImageKit (see below)... I'm not able to make it load
the libsquashfuse I already packaged. CMake refuses to find it.

I could use a hand to make this package work or at least some help with CMake,
because I'm not used to work with it and I know some of you have dealt with it
much more than what I did.

Can anyone help me with this?


This is the current file I'm using to test all this:


(define-module (appimage)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module ((srfi srfi-1) #:hide (zip))
  #:use-module (guix git-download)
  #:use-module (guix build-system cmake)
  #:use-module (guix build-system gnu)
  #:use-module (guix packages)
  #:use-module (gnu packages)
  #:use-module (gnu packages wget)
  #:use-module (gnu packages vim)
  #:use-module (gnu packages freedesktop)
  #:use-module (gnu packages autotools)
  #:use-module (gnu packages pkg-config)
  #:use-module (gnu packages gtk)
  #:use-module (gnu packages gnome)
  #:use-module (gnu packages boost)
  #:use-module (gnu packages tls)
  #:use-module (gnu packages compression)
  #:use-module (gnu packages commencement)
  #:use-module (gnu packages backup)
  #:use-module (gnu packages version-control)
  #:use-module (gnu packages linux))


(define-public squashfuse
  (package
(name "squashfuse")
(version "0.1.104")
(source (origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/vasi/squashfuse.git;)
 (commit version)))
  (sha256
   (base32
"1j9phqgg3xqfyk94h1vcl5gkyhdfnglmzwva2vvp5ri07b76y3h4"
(build-system gnu-build-system)
(native-inputs
  `(("automake" ,automake)
("autoconf" ,autoconf)
("pkg-config" ,pkg-config)
("libtool" ,libtool)))
(inputs
 `(("fuse" ,fuse)))
(synopsis "FUSE filesystem to mount squashfs archives")
(description "Squashfuse lets you mount SquashFS archives in user-space. It
supports almost all features of the SquashFS format, yet is still fast and
memory-efficient.")
(home-page "https://github.com/vasi/squashfuse;)
(license license:bsd-2)))


(define-public xdg-utils-cxx
  (package
(name "xdg-utils-cxx")
(version "1.0.1")
(source (origin
  (method git-fetch)
  (uri (git-reference
  (url "https://github.com/azubieta/xdg-utils-cxx.git;)
  (commit (string-append "v" version))
  (recursive? #t)))
  (sha256
(base32
  "00nvn00z1c7xvpb7xjm6i9i62hchg3nandxw487g6dallv378hw4"
(build-system cmake-build-system)
(arguments
  `(#:configure-flags
 (list "-DXDG_UTILS_SHARED=ON")))
(synopsis "Implementation of the FreeDesktop specifications to be used in 
C++ projects")
(description "Implementation of the Free Desktop Standards in C++.  This is
a project was started to fulfill the need of a reliable implementations of such
standards in the AppImage project. It is totally standalone and only depends on
the standard c++ libraries (stdlib).")
(home-page "https://github.com/azubieta/xdg-utils-cxx;)
(license license:expat)))

(define-public libappimage
  (package
(name "libappimage")
(version "1.0.3")
(source (origin
  (method git-fetch)
  (uri (git-reference
 (url "https://github.com/AppImage/libappimage.git;)
 (commit (string-append "v" version))
 (recursive? #t)))
  (sha256
   (base32
"1b8kkypzvf12xp25lgcrqrflhj9j8kgwd39za4w90vx3q0c2lpa0"
(build-system cmake-build-system)
(arguments
 `(#:configure-flags
   (list "-DBUILD_TESTING:bool=False" ; Tests are broken
 "-DUSE_SYSTEM_BOOST=On"
 "-DUSE_SYSTEM_LIBARCHIVE=On"
 "-DUSE_SYSTEM_XZ=On"
 "-DUSE_SYSTEM_XDGUTILS=On"
 "-DUSE_SYSTEM_SQUASHFUSE=On")
   #:tests? #f))  ; Tests are broken
(native-inputs
 `(("automake" ,automake)
   ("autoconf" ,autoconf)
   ("git" ,git)
   ("libtool" ,libtool)
   ("pkg-config" ,pkg-config)))
(inputs
 `(("boost" ,boost)
   ("cairo" ,cairo)
   ("desktop-file-validate" ,desktop-file-utils)
   ("fuse" ,fuse)
   ("libarchive" ,libarchive)
   ("librsvg" ,librsvg)
   ("squashfuse" ,squashfuse)
   ("wget" ,wget)
   ("xdg-utils-cxx" ,xdg-utils-cxx)
   ("xxd" ,xxd)
   ("xz" ,xz)))
(synopsis "Implements functionality for dealing with AppImage files")
(description "Library for dealing with AppImage files.")
(home-page "https://github.com/AppImage/libappimage/;)
(license license:expat)))


(define-public appimagekit
  (package
(name "appimagekit")
(version "13")

Re: Thursday 18th: ‘core-updates-frozen’ sprint!

[zimoun]

Hi Ludo,

On Mon, 22 Nov 2021 at 14:21, Ludovic Courtès  wrote:

> >> … there’s probably more.

Julia is annoying,  Because the test suite sometimes passes, sometimes
not.  Well, I am not even sure it is the same part that fails.  I
notice that "guix build julia --without-tests=julia" then "guix build
julia" tends to works. :-)

Any idea to debug this?


Cheers,
simon

Re: Replacing polkit by polkit-duktape on core-updates-frozen ?

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer  skribis:

> +;;; Define a top level polkit variable that can be built on any of the
> +;;; supported platforms.  This is to work around the fact that our
> +;;; mrustc-bootstrapped rust toolchain currently only supports the x86_64
> +;;; architecture...
> +(define-public polkit
> +  (if (string-prefix? "x86_64"
> +  (or (%current-system) (%current-target-system)))
> +  polkit*
> +  polkit-duktape))

This would instead have to be a macro, similar to the ‘pkg-config’
macro, so that (%current-system) is checked at the right time rather
than when loading this module.

However, since the only different between polkit and polkit-duktape
(IIUC) is that an extra patch is applied for the latter, I would instead
suggest adding a conditional build phase that applies the patch on
non-x86_64 systems.

How does that sound?

Ludo’.

Re: Using G-Expressions for public keys (substitutes and possibly more)

[Ludovic Courtès]

Hi,

Liliana Marie Prikler  skribis:

> I think we would probably want to improve on this end in the guile-
> gcrypt module, i.e. have a public-key "constructor" that returns a
> canonical-sexp and so on.  WDYT?

I don’t find it very compelling given there’s already
‘sexp->canonical-sexp’ & co.  WDYT?

Ludo’.

Re: [core-updates-frozen] Different variants of Python packages in the same profile?

[Ludovic Courtès]

Hi,

Ricardo Wurmus  skribis:

> on the core-updates-frozen branch we have a minor problem:
> python-numpy has been upgraded to 1.21, but python-numba can only 
> be built with 1.20[1].  I added python-numpy-1.20 and made
> python-numba use it.

How big of an effort would it be to modify numba so that it works with
1.21?  With luck, that effort has already been made upstream?

Ludo’.

Re: Flag day for simplified package inputs

[Ludovic Courtès]

Hi,

Jelle Licht  skribis:

> When applying this and future bulk changes, could we perhaps list the
> specific commits (+ commented shortlog line) in a file. To clarify, if
> we were to store these commits in `.git-blame-ignore-revs', later on we
> can instruct users to run:
>
> git config blame.ignoreRevsFile .git-blame-ignore-revs
>
> to ignore the bulk change for git blame purposes.

Sounds like a good idea.  There’s no particular convention regarding the
file name of ‘.git-blame-ignore-revs’, right?

> It seems like a maintainable way to mitigate some of the (IMO) major
> disadvantages that we bring in by applying bulk changes.

The disadvantage of changes is that they introduce changes, indeed.  :-)

But really, I think we should be able to find relevant hints as comments
and use VCS history more as a last resort.

Thanks,
Ludo’.

Re: Thursday 18th: ‘core-updates-frozen’ sprint!

[Ludovic Courtès]

Hi!

Ricardo Wurmus  skribis:

> Ludovic Courtès  writes:
>
>> … there’s probably more.
>
> I noticed that we’re still using PYTHONPATH in a bunch of places, even
> though we should now be using GUIX_PYTHONPATH.  I’ve been slowly
> hacking away at the remaining offenders.

I see you fixed a bunch of them, yay!

> That said, it’s amazing how much progress we made in so little time!
> I’ll continue to prioritize core-updates-frozen for the next few days.

Yes, let’s do that.  I’ve fixed things I’ll probably never ever use,
like trytond.  A call to everyone though: make sure your favorite
packages work because you can’t necessarily count on others to test it
and fix it.  :-)

I’ve noticed Python packages still failing so if you rely on Python,
better be safe than sorry.

>>> Nov. 23th will mark Guix’s 9th birthday.  Wouldn’t it be great to
>>> have
>>> that branch finally merged in ‘master’ by then?
>>
>> Sounds doable, let’s keep up the good work!
>
> I hope we can make it!

Yup, let’s see!

Ludo’.

Re: default tar format for "make dist" and patch file length

[Ludovic Courtès]

Hi!

Maxim Cournoyer  skribis:

> Reading the Automake manual (info (automake) List of Automake options) I
> stumbled on this:
>
> ‘filename-length-max=99’
>  Abort if file names longer than 99 characters are found during
>  ‘make dist’.  Such long file names are generally considered not to
>  be portable in tarballs.  See the ‘tar-v7’ and ‘tar-ustar’ options
>  below.  This option should be used in the top-level ‘Makefile.am’
>  or as an argument of ‘AM_INIT_AUTOMAKE’ in ‘configure.ac’; it will
>  be ignored otherwise.  It will also be ignored in sub-packages of
>  nested packages (*note Subpackages::).
>
> This makes me think that Automake is simply configured out of the box to
> keep the file names as portable as possible (it doesn't mean it uses
> tar-v7 itself, IIUC, though I haven't checked).

Oh, looks like we could add the ‘tar-ustar’ option and be done with it
(info "(automake) List of Automake options"):

 ‘tar-ustar’ selects the ustar format defined by POSIX 1003.1-1988.
 This format is old enough to be portable: As of 2018, it is
 supported by the native ‘tar’ command on GNU, FreeBSD, NetBSD,
 OpenBSD, AIX, HP-UX, and Solaris, at least.  It fully supports
 empty directories.  It can store file names with up to 256
 characters, provided that the file name can be split at directory
 separator in two parts, first of them being at most 155 bytes long.
 So, in most cases the maximum file name length will be shorter than
 256 characters.

Any objections against the patch below?

diff --git a/configure.ac b/configure.ac
index 84592f6041..341cff8fbd 100644
--- a/configure.ac
+++ b/configure.ac
@@ -8,7 +8,7 @@ AC_INIT([GNU Guix],
   [https://www.gnu.org/software/guix/])
 AC_CONFIG_AUX_DIR([build-aux])
 
-AM_INIT_AUTOMAKE([1.14 gnu silent-rules subdir-objects \
+AM_INIT_AUTOMAKE([1.14 gnu tar-ustar silent-rules subdir-objects \
  color-tests parallel-tests -Woverride -Wno-portability])
 
 # Enable silent rules by default.

I tried it and there’s no tar warning and the file looks just fine,
including the long file names:

--8<---cut here---start->8---
$ make dist -j5
[…]
make[2]: Leaving directory '/home/ludo/src/guix'
tardir=guix-1.3.0.8612-7cad44-dirty && tar --format=ustar -chf - "$tardir" | 
eval GZIP= gzip --best -c >guix-1.3.0.8612-7cad44-dirty.tar.gz
make[1]: Leaving directory '/home/ludo/src/guix'
if test -d "guix-1.3.0.8612-7cad44-dirty"; then find 
"guix-1.3.0.8612-7cad44-dirty" -type d ! -perm -200 -exec chmod u+w {} ';' && 
rm -rf "guix-1.3.0.8612-7cad44-dirty" || { sleep 5 && rm -rf 
"guix-1.3.0.8612-7cad44-dirty"; }; else :; fi
$ tar tvf guix-1.3.0.8612-7cad44-dirty.tar.gz |grep patches/xplanet
-rw-r--r-- ludo/users 1252 2020-04-06 00:14 
guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-libimage_gif.c.patch
-rw-r--r-- ludo/users  765 2020-04-06 00:14 
guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-xpUtil-Add2017LeapSecond.cpp.patch
-rw-r--r-- ludo/users 6990 2020-04-06 00:14 
guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-cxx11-eof.patch
-rw-r--r-- ludo/users  783 2020-04-06 00:14 
guix-1.3.0.8612-7cad44-dirty/gnu/packages/patches/xplanet-1.3.1-libdisplay_DisplayOutput.cpp.patch
--8<---cut here---end--->8---

> Seems a good thing to be as portable as can be, especially since 200
> chars patch file names wouldn't look good in the sources anyway ;-).

I don’t think we care about “portability” as understood in the context
of Automake, which roughly translates to “support SunOS 2 and HP-UX’s
1991 ‘tar’ implementation”.  :-)

Thanks,
Ludo’.