Re: [PATCH] ext2fs: Update to upstream Hurd-reserved xattr index for "gnu.*".

[Samuel Thibault]

Hello,

Jan Nieuwenhuizen, le mer. 13 mai 2020 14:04:04 +0200, a ecrit:
> So WDYT, is my patch for the Hurd (the first message in this thread)

Applied, thanks!

> Any ideas or suggestions on my Linux patch?

I would say not bother introducing a configuration variable, and just
include xattr_hurd.o in ext4-y. This is probably a matter of a few
hundred bytes only, it's not worth extending the already-long list of
options.

Samuel

Re: core-updates all over again :)

[Marius Bakke]

Christopher Baines  writes:

> Hey,
>
> So, given that core-updates was recently merged, I guess it's time to
> start the branch again?
>
> I've got two patches in mind, [1] for parallel xz compression during the
> unpack phase if the build is happening in parallel, and [2] to enable
> p11-kit in gnutls.
>
> 1: https://issues.guix.info/issue/33643
> 2: https://issues.guix.info/issue/40654

For [2], you could define a 'gnutls+p11-kit' and use that in libcacard
for now, instead of having to wait N months for the branch to start
rolling (during which time things may change in libcacard or GnuTLS).

> There's also updating Ruby to 2.7 which can at least be considered for
> core-updates, I've seen some 2.7 incompatibilities in various Ruby
> libraries, but it's worth looking at.

Maybe we should have a 'package-with-ruby27' procedure, similar to
'package-with-python2'?  Then we don't need to wait for core-updates to
try this either.  :-)

> The above patches could do with a bit more review, but is there any
> formalities in terms of getting core-updates going again?

The job specification needs to be re-added on Cuirass, but other than
that it should be "safe to push".


signature.asc
Description: PGP signature

Re: [FONTS]: Media Type Specification Changed.

[Raghav Gururajan]

Hi Tobias!

> Thanks!  I'm better than I've been, don't worry.
> 
>> That change has been discussed during review and was only merged after I
>> gave the fact of about fontconfig. :-)

I am glad.

> LGTM, apart from one nitpick.
> 
> Both /share/fonts/{true,open}type are extremely common on other
> distributions, while /share/fonts/webopen would be unique to Guix. The
> de facto ’standard’ for (the much newer) WOFF seems to be, well,
> /share/fonts/woff.  So despite my previous comments about avoiding
> acronyms, I'll keep the latter.  We can always revisit this later.
> 
> Applied as 7d426c5b0e27cc3e72c6e12e07b8c42055cedba0.  Thanks!

Thank you!

I was wondering if you could share your thoughts on #40708?

Regards,
RG.





signature.asc
Description: OpenPGP digital signature

core-updates all over again :)

[Christopher Baines]

Hey,

So, given that core-updates was recently merged, I guess it's time to
start the branch again?

I've got two patches in mind, [1] for parallel xz compression during the
unpack phase if the build is happening in parallel, and [2] to enable
p11-kit in gnutls.

1: https://issues.guix.info/issue/33643
2: https://issues.guix.info/issue/40654

There's also updating Ruby to 2.7 which can at least be considered for
core-updates, I've seen some 2.7 incompatibilities in various Ruby
libraries, but it's worth looking at.

The above patches could do with a bit more review, but is there any
formalities in terms of getting core-updates going again?

Thanks,

Chris


signature.asc
Description: PGP signature

Re: best practise between git-fetch vs url-fetch?

[Tobias Geerinckx-Rice]

Les Guix,

Brice Waegeneire 写道：

Git references being content-addressed is important


That's only tautologically true if we limit ourselves to using raw 
commit hashes instead of the more common (because far more 
practical) named tags.  I obviously don't think we should do that.


If we don't, I fail to see how our git-fetches are 
‘content-addressed’ in a way that url-fetches supposedly aren't. 
Git tags are even easier to modify in place than tarballs, not 
harder.



--with-commit


Yes, this is niice.  ♥

For the sake of argument¹, though, so is --with-source=released and supported upstream tarball dot tar>.


Somehow erasing that hard distinction is the real winning move.

Kind regards,

T G-R

[1]: Obligatory .


signature.asc
Description: PGP signature

Re: best practise between git-fetch vs url-fetch?

[Leo Famulari]

On Wed, May 13, 2020 at 03:08:26AM +0200, zimoun wrote:
> Based on these 2 messages [1,2], what is the consensus between
> git-fetch and url-fetch?

Do we need a consensus? Sometimes it's enough for the reviewer to 1)
bite their tongue or 2) fix before pushing. Often it's a matter of taste
and it is beneficial to not second-guess the patch author too much, to
not hurt their confidence.

I think we should try to avoid a situation where we have to bootstrap
Git. We do have git-minimal, which works for now. Libgit2 recently
started releasing tarballs, so that could be an option, too.

Another point for url-fetch is that Git's transition from SHA1 to SHA256
identifiers may be easy for us, or it may not be. We don't know yet.

Re: [FONTS]: Media Type Specification Changed.

[Tobias Geerinckx-Rice]

Raghav,

Raghav Gururajan 写道：
Oh! How are you doing now? Let me know if you need anything. I 
can try

and help in any way plausible. :-)


Thanks!  I'm better than I've been, don't worry.

That change has been discussed during review and was only merged 
after I

gave the fact of about fontconfig. :-)


All right!

I understand. I should have mentioned in the log. But it was not 
sneaky.


I tried to be very clear that I didn't actually think so.  Sorry 
if I failed.


But yeah, I should add that to my workflow of making 
comments/logs.


Sweet.

Agreed! I do not like inconsistencies either. Please find the 
patch

attached with this email.


LGTM, apart from one nitpick.

Both /share/fonts/{true,open}type are extremely common on other 
distributions, while /share/fonts/webopen would be unique to Guix. 
The de facto ’standard’ for (the much newer) WOFF seems to be, 
well, /share/fonts/woff.  So despite my previous comments about 
avoiding acronyms, I'll keep the latter.  We can always revisit 
this later.


Applied as 7d426c5b0e27cc3e72c6e12e07b8c42055cedba0.  Thanks!

Kind regards,

T G-R

[0]: 


signature.asc
Description: PGP signature

Re: Towards a graphical installer?

[Mathieu Othacehe]

Hello Florian,

> I suppose keyboard input method (IME) support is a reason why someone
> might wish to use an Xorg-based installer.  Are there other reasons?
> What are the reasons for a real desktop environment; is the goal to
> offer a live image?  Previously I would have thought a virtual console
> is more likely to work everywhere, but kmscon has the same
> requirements as Xorg and without kmscon we’d lose i18n.

I think switching to a DE based installer could:

* Allow the user to browse for some questions or contact #guix while in
  the installation process.

* Bring better control of network connection, with a dedicated tool such
  as NetworkManager applet.

* Make easier to add accessibility support to the installer.

Regarding i18n, the current mechanism would indeed be broken. However,
switching to setxkbmap or any GNOME API should not be too hard.

Thanks,

Mathieu

Re: Towards a graphical installer?

[Mathieu Othacehe]

Hello Tobias and Jonathan,

> Are you, for example, able to connect to a Wi-Fi network the Gnome way (not
> using the installer), without a Gnome authentication dialogue popping up that
> doesn't understand the notion of ‘no password’?  I had to open a terminal and
> ‘passwd’ myself out of that to continue.

No, I kept connman as the installer network-manager.

> Last time I brought this up someone mentioned using Calamares[0]. Not that I
> think that's a good idea, but it's another data point for discussion :-p

My main concern with Calamares, Anaconda & friends is that we will
inevitably have to write some Python/C++ to adapt it to Guix System
installation. Now that we have Guile code covering the whole
installation and associated automated tests, I would prefer not to use
those external tools.

> As much as I dislike Gnome, I think we should first try to use the (most
> likely) default desktop during the installation, and I do think that's
> currently Gnome.

I agree that even though GNOME is heavy and hard to configure, its also
probably to most popular DE.

To adapt the current installer to GNOME, we would need to:

* Switch from connman to NetworkManager.
* Add DBus bindings to control NetworkManager.
* Use 'setxkbmap' or a GNOME API to handle keyboard layout switching.
* Open the info page somewhere else than on TTY2, maybe in a
Web-browser.

Thanks,

Mathieu

Re: [PATCH] ext2fs: Update to upstream Hurd-reserved xattr index for "gnu.*".

[Jan Nieuwenhuizen]

Samuel Thibault writes:

Hello Samuel,

> Jan Nieuwenhuizen, le mar. 12 mai 2020 16:12:34 +0200, a ecrit:
>> setfattr --name=gnu.translator --value='/hurd/pflocal\0' 
>> /mnt/servers/socket/1
>
> man setfattr says
>
> If the given string is enclosed in double quotes, the inner string is
> treated as text.  In that case, backslashes and double quotes have
> special meanings and need to be escaped by a preceding backslash.
>
> so I guess it needs 
>
> setfattr --name=gnu.translator --value='"/hurd/pflocal\0"' 
> /mnt/servers/socket/1
>
> to actually interpret \0

Yes, that's it; thank you!

So I've now managed to create a vm-image using Guix that boots the Hurd
straight into our Guile RC script, without the need to use Bash for
that.  See


https://gitlab.com/janneke/guix/-/commit/3b4c0dcda783e8c866be154ab46ec95fb581f08f
./pre-inst-env guix system vm-image --target=i586-pc-gnu 
gnu/system/examples/bare-hurd.tmpl

and

wget https://lilypond.org/janneke/hurd/hurd-xattr.img.gz
gunzip hurd-xattr.img.gz
guix environment --ad-hoc qemu -- qemu-system-i386 -enable-kvm \
 -snapshot -hda hurd-xattr.img -m 2G \
 -device rtl8139,netdev=net0 -netdev 
user,id=net0,hostfwd=tcp:127.0.0.1:10022-:

So WDYT, is my patch for the Hurd (the first message in this thread)

https://lists.gnu.org/archive/html/bug-hurd/2020-05/msg00016.html

OK to apply now?  Any ideas or suggestions on my Linux patch?

>> --8<---cut here---start->8---
>> fsck --yes --force /
>> fsysopts / --writable
>> mv /servers/socket/1 /servers/socket/1-linux
>> touch /servers/socket/1
>> setfattr --name=gnu.translator --value='/hurd/pflocal\0' /servers/socket/1
>
> Note that glibc's setxattr, i.e. _hurd_xattr_set, translates that
> into a __file_set_translator() RPC call. Did you pass the proper option
> to make ext2fs record the translator as xattr instead of passive record?

Yes...

>> And I guess there must be an incompatibility between Linux and the Hurd
>> in how setfattr embeds the xattr attributes into the file system.
>> 
>> How to best "diff" this aspect in the file system; how to proceed?
>
> debugfs can be used for that.

Right, thanks.  I was looking for something like that.

>> Inspired by Shengyu's GSoC code that simply seemed to use fprintf for
>> debbugging, I tried adding some debug printing in inode.c
>> 
>> fprintf (stderr, "gnu.translator[%d,%d]=%s\n", datalen, strlen (*namep), 
>> *namep);
>
> Printf is the simplest way to make sure things are happening the way one
> wants, yes. Note however that in the case of translators even the output on
> stderr is buffered, so you need to flush it with fflush(stderr). Even
> safer is to use snprintf + mach_print().
>
>> but that does not seem to work,
>
> More precisely?
> I'll assume "does not show up".

(that was more precisely exactly what I meant to ask)

> If your print doesn't show up, try to put a print in other places which
> are definitely to be called such as in diskfs_user_read_node(). If those
> come up, then it means the code you put your prints is not even called,
> so put prints in code you thought was calling it etc. up to the RPC that
> you thought would be called, then jump to libc which was supposed to be
> making the RPC call, etc.

Thank you!  It took me a while to find the right setfattr curse so I
dabbled in here some more and can confirm that combining your
instructions, e.g., like so

--8<---cut here---start->8---
diff --git a/ext2fs/inode.c b/ext2fs/inode.c
index a2e804b9..f4e29eb5 100644
--- a/ext2fs/inode.c
+++ b/ext2fs/inode.c
@@ -700,6 +700,9 @@ diskfs_get_translator (struct node *np, char **namep, 
unsigned *namelen)
   void *transloc;
   struct ext2_inode *di;
 
+  fprintf (stderr, "diskfs_get_translator\n");
+  fflush (stderr);
+
   if (sblock->s_creator_os != EXT2_OS_HURD)
 return EOPNOTSUPP;
--8<---cut here---end--->8---
 
"just work".  That's helpful knowledge to have anyway.

Greetings,
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Verification Builds for Guix

[zimoun]

Dear,

I do not know if it is relevant and if it completes what janneke already said.


On Wed, 13 May 2020 at 08:54, Jan Nieuwenhuizen  wrote:

> > Checking out the packages part of the website I found that there is at
> > least an CI creating binaries. Are those also offered to users?

The build farm (by default) is reachable at [1] and it offers binary
substitutes.  Another entry point is Guix Data Service [1] which
collects data about packages, builds, branches, etc.; for example the
history of versions [3] or the status of builds [4].  Moreover, there
is a recent proposal: build-coordinator [5].

[1] https://ci.guix.gnu.org/
[2] http://data.guix.gnu.org/
[3] https://data.guix.gnu.org/repository/1/branch/master/package/git
[4] 
https://data.guix.gnu.org/repository/1/branch/master/package/git/output-history?output=out=armhf-linux=none
[5] https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00323.html


> This hash includes the transitive dependencies and can be calculated
> without compiling anything.  The "binary package", once built, will be
> installed using this hash, e.g.:
> "/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2".

This hash identifies the package and the transitive dependencies, so
the hash is an unique identifier (modulo hash collisions which are
another story).
The same hash should produce the same binary or there is source of
non-determinism.  That's why "guix build --check" rebuilds and compare
the two builds.

I do not know what is the policy on the build farm ci.guix.gnu.org
about checking the rebuild.


> Guix does have the "guix challenge" command
>
> --8<---cut here---start->8---
> guix challenge --help
> Usage: guix challenge [PACKAGE...]
> Challenge the substitutes for PACKAGE... provided by one or more servers.
>
>   --substitute-urls=URLS
>  compare build results with those at URLS
> --8<---cut here---end--->8---
>
> now the trick is, to get "someone" to run that on an interesting portion
> of the archive...and to report the results in some common format.

I am not aware of such initiative.  Which should be really cool.

On the other hand, using this information of challenging the local
builds against remote builds would allow to share /gnu/store/; other
said full distributed substitutes mechanism.



Best regards,
simon

Re: Towards a graphical installer?

[pelzflorian (Florian Pelz)]

On Wed, May 13, 2020 at 09:20:36AM +0200, Mathieu Othacehe wrote:
> I think switching to a DE based installer could

Every reason you list is a good reason.

> * Allow the user to browse for some questions or contact #guix while in
>   the installation process.

Hmm what I don’t like about heavyweight live images is that there are
so many possibilities and they make so many decisions on what is the
default.  Currently I can guix install the programs I actually want.
Oh well.  Maybe not preinstall all those apps, be it an IRC client, a
Web browser, GNUnet or whatever we like, but instead have the user
choose to install them?

Regards,
Florian

Re: best practise between git-fetch vs url-fetch?

[Brice Waegeneire]

Hello,

On 2020-05-13 01:08, zimoun wrote:

Based on these 2 messages [1,2], what is the consensus between
git-fetch and url-fetch?


I was hoping that some one bring this up, thanks.


Pushing to SWH when linting appears to me winning the pros/cons.  Even
if SWH should eventually fetch http://guix.gnu.org/sources.json soon.
And the other big pros from my point of view is the content-addressed
Git references.


Git references being content-addressed is important because it
make it more difficult for a lazy upstream developer to replace a
tarball in place -because it was somehow broken and they didn't
wanted to bump the version number- and broke a package.  Instead
with git, to broke a package in similar way (with a hash mismatch)
that developer would have to do “git push --force” which is much
more frowned upon since it will affect all the users of that git repo.

An other argument in recommending the 'git-fetch' method is that it
facilitate the use of “guix build --with-commit”:
1. You don't need to find out the git-url of the package,
2. Nor finding which dependencies are missing compared to a tarball
   build (often it's autoconf, libtool, & co.),
3. You don't need to manually tweak phases relating to autoconf and
   the like.
All in all recommending 'git-fetch' make “--with-commit” really more
useful.  This feature is one of my favorite from Guix since it make
testing a patch from an upstream developer really easy, eg.:
“guix build picom --with-commit=picom=vNext”.


Well, does it make sense to state on a recommended method?


It does, it avoid having to discuss it with each new contributor
and avoid noise in patches about changing the source's method based
on each developer preference (I'm personally guilty of that).


[1] https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00091.html
On Fri, 6 Mar 2020 at 18:41, Marius Bakke  wrote:


url-fetch requires less bandwidth, and does not depend on 'git'.

Though the most important distinction is that uploaded releases
sometimes contain pre-processed sources (e.g. documentation) that need
additional dependencies or scripts when building from the raw 
repository
(this is why you often need to add autoconf, libtool & friends as 
inputs

when building Autotools projects from git).

I don't know whether there is a difference between the uploaded fmt
zipball and the git repository.



[2] https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00189.html
On Wed, 11 Mar 2020 at 15:39, Ludovic Courtès  wrote:


Other considerations:

  - Bandwidth requirement for source code downloads has never been a
criterion so far.

  - Git references are nice because they’re (roughly) 
content-addressed.


  - ‘guix lint -c archival’ archives Git references on Software
Heritage; it does not archive tarballs (though SWH will do it
for us eventually.)


Cheers,
- Brice

Re: Towards a graphical installer?

[Pierre Neidhardt]

dftxbs3e  writes:

> On 5/12/20 9:09 AM, Pierre Neidhardt wrote:
>> Errr... Sorry, I was distracted yesterday, the NLNet grant is actually
>> for a graphical _package manager_, not an installer! :p
>> 
>> That said, I'm interested in helping with it too! :)
>> 
>
> Really?! We have that?? Wonderful!! That would be the YaST thing I was
> talking about!!

Not yet, but hopefully soon :)

To clarify, what are you expecting from a "YaST thing"?

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Verification Builds for Guix

[Jan Nieuwenhuizen]

Paul Spooren writes:

[cc: guix-devel]

Hello Paul,

> I've used the last week a bit to work on some kind of verification build
> collector. Essentially a scraper of rebuild results. Meaning, an official 
> binary
> provided by a project is tried to rebuild.

Okay...

> This works already for Archlinux and OpenWrt[0], now I wanted to know if the
> same would make sense for Guix.

Possibly...I'm not sure, cc'ing guix-devel :-)

> Checking out the packages part of the website I found that there is at
> least an CI creating binaries. Are those also offered to users?

Yes, Guix (like Nix) uses a mechanism that enables (encourages?) users
to always build everything from source.  The result of a package build,
a "binary package" is uniquely identified by a hash of its inputs,
including its build recipe, e.g.,
"18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2".

This hash includes the transitive dependencies and can be calculated
without compiling anything.  The "binary package", once built, will be
installed using this hash, e.g.:
"/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2".

A user can opt-in to using "binary substitutes".  In that case when a
installing a package, before the package build actually starts, Guix
checks if a binary is already available from a substitute server using
this hash.  If a binary is available it simply (well...keys/trust etc.)
downloads the build result.  In my case it checks for these in order


http://banaan.local:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2

http://janneke.lilypond.org:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2

http://kluit.dezyne.org:8181/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2

https://ci.guix.gnu.org/nar/lzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2

> If so a rebuilder of exactly these images would be interesting to
> integrate in the rebuild-collector.

Okay...so be clear on this; Guix (nor Nix) have the concept of
REbuilders; all builders are equal: the central CI builder is not "more
equal" than any other ;-)

Guix does have the "guix challenge" command

--8<---cut here---start->8---
guix challenge --help
Usage: guix challenge [PACKAGE...]
Challenge the substitutes for PACKAGE... provided by one or more servers.

  --substitute-urls=URLS
 compare build results with those at URLS
--8<---cut here---end--->8---

now the trick is, to get "someone" to run that on an interesting portion
of the archive...and to report the results in some common format.

Looking at

> [0]: https://rebuild.aparcar.org/

I think it could work, although for Guix I think it would be more
natural to consider the local build to be the the "original" to compare
binary substitute servers against?  Maybe the format could be more
symmetrical?

Greetings,
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com