Re: advanced?

[Development of GNU Guix and the GNU System distribution.]

Ludovic Courtès  writes:

> Hello,
>
> (Cc: Luis, for the web site design.)
>
> Simon Josefsson  skribis:
>
>> Ludovic Courtès  writes:
>>
>>> Or if we do want to explain more, then perhaps we need a list of
>>> features that would also include things like Docker/VM image generation,
>>> declarative home environments, etc.  But that’s broader topic.
>>
>> Yes, that makes sense.  I'm not the best person to summarize it, but
>> starting pointers if someone wants to take it further:
>>
>> * Dedication to free software goals and the GNU community
>>
>> * Shepherd init system written in Guile
>>
>> * Declarative stateless system configurations
>>
>> * Transactional upgrades and roll-backs
>>
>> * Reproducible build environments
>>
>> * Designed towards bootstrappable builds
>>
>> Maybe this fits better directly in the Introduction section of the
>> manual?  https://guix.gnu.org/manual/en/html_node/Introduction.html
>
> I guess we should rework the “Introduction” and “Features” sections,
> which were written in the early days.
>
> The points you list above are a great starting point, and I guess that
> would also be a good fit for the front page; currently there’s no
> “feature list” there.  That old “Guix in action” video also ought to be
> replaced.

Reflecting on the feature list, I think we should mention that Guix is a
_rolling_ distribution and package manager, and maybe explain what that
means.  I don't think this is clear from the web site or manual now, but
I may be missing it.

Perhaps the release and update model of Guix could use some dedicated
new documentation?  The relationship between the rolling master branch,
the core-updates branch, the security graft mechanism, the substitute
build servers and the versioned installer releases is not terribly clear
to me as a new user, and having an understanding of these concepts helps
to make contributions.  I have a feeling there may be more nuances that
are useful to know about that I'm not familiar with; for example, the
intended use of the version-X.Y.Z branches.

/Simon


signature.asc
Description: PGP signature

Re: advanced?

[Julien Lepiller]

Already fixed in weblate for French. Will push the update shortly :)

Le 9 janvier 2023 12:12:27 GMT+01:00, "Ludovic Courtès"  a écrit :
>Hey,
>
>A heads-up for Julien and the translators:
>
>Ludovic Courtès  skribis:
>
>> Simon Josefsson via "Development of GNU Guix and the GNU System
>> distribution."  skribis:
>>
>>> From aac8f6d1fb382b9f9120b7cd51dc80e8ef07cc03 Mon Sep 17 00:00:00 2001
>>> From: Simon Josefsson 
>>> Date: Sat, 26 Nov 2022 22:35:15 +0100
>>> Subject: [PATCH] website: Reduce use of 'advanced' term.
>>>
>>> ---
>>>  website/apps/base/templates/about.scm | 4 ++--
>>>  website/apps/base/templates/home.scm  | 6 +++---
>>>  2 files changed, 5 insertions(+), 5 deletions(-)
>>
>> It’s been a month and it looks like there wasn’t any opposition to this
>> change, so I went ahead and pushed it.
>
>Unsurprisingly, removing the one word invalidated all translations,
>as can be seen at the top of the front page:
>
>  <https://guix.gnu.org/fr/>
>
>Ludo’.

Re: advanced?

[Ludovic Courtès]

Hey,

A heads-up for Julien and the translators:

Ludovic Courtès  skribis:

> Simon Josefsson via "Development of GNU Guix and the GNU System
> distribution."  skribis:
>
>> From aac8f6d1fb382b9f9120b7cd51dc80e8ef07cc03 Mon Sep 17 00:00:00 2001
>> From: Simon Josefsson 
>> Date: Sat, 26 Nov 2022 22:35:15 +0100
>> Subject: [PATCH] website: Reduce use of 'advanced' term.
>>
>> ---
>>  website/apps/base/templates/about.scm | 4 ++--
>>  website/apps/base/templates/home.scm  | 6 +++---
>>  2 files changed, 5 insertions(+), 5 deletions(-)
>
> It’s been a month and it looks like there wasn’t any opposition to this
> change, so I went ahead and pushed it.

Unsurprisingly, removing the one word invalidated all translations,
as can be seen at the top of the front page:

  <https://guix.gnu.org/fr/>

Ludo’.

Re: advanced?

[Ludovic Courtès]

Hello,

(Cc: Luis, for the web site design.)

Simon Josefsson  skribis:

> Ludovic Courtès  writes:
>
>> Or if we do want to explain more, then perhaps we need a list of
>> features that would also include things like Docker/VM image generation,
>> declarative home environments, etc.  But that’s broader topic.
>
> Yes, that makes sense.  I'm not the best person to summarize it, but
> starting pointers if someone wants to take it further:
>
> * Dedication to free software goals and the GNU community
>
> * Shepherd init system written in Guile
>
> * Declarative stateless system configurations
>
> * Transactional upgrades and roll-backs
>
> * Reproducible build environments
>
> * Designed towards bootstrappable builds
>
> Maybe this fits better directly in the Introduction section of the
> manual?  https://guix.gnu.org/manual/en/html_node/Introduction.html

I guess we should rework the “Introduction” and “Features” sections,
which were written in the early days.

The points you list above are a great starting point, and I guess that
would also be a good fit for the front page; currently there’s no
“feature list” there.  That old “Guix in action” video also ought to be
replaced.

Luis, do you have ideas as to how we could present things?

Thanks,
Ludo’.

Re: advanced?

[Ludovic Courtès]

Hi Simon and all,

Simon Josefsson via "Development of GNU Guix and the GNU System
distribution."  skribis:

> From aac8f6d1fb382b9f9120b7cd51dc80e8ef07cc03 Mon Sep 17 00:00:00 2001
> From: Simon Josefsson 
> Date: Sat, 26 Nov 2022 22:35:15 +0100
> Subject: [PATCH] website: Reduce use of 'advanced' term.
>
> ---
>  website/apps/base/templates/about.scm | 4 ++--
>  website/apps/base/templates/home.scm  | 6 +++---
>  2 files changed, 5 insertions(+), 5 deletions(-)

It’s been a month and it looks like there wasn’t any opposition to this
change, so I went ahead and pushed it.

Thanks,
Ludo’.

Re: Advanced distribution

[Ryan Prior]

Hi Octavio! We had a discussion about this last month, and we might make some 
changes to make it clearer what "advanced" means (or perhaps change the 
wording.)

Here's a link to that discussion in the list archive: 
https://lists.gnu.org/archive/html/guix-devel/2022-11/msg00298.html

Cheers,
Ryan

P.S. apropos of your email header: this email thread will also be visible on 
the public Guix archive. For sensitive issues please consider sending to 
guix-maintain...@gnu.org
--- Original Message ---
On Saturday, December 10th, 2022 at 3:08 AM, Engineer Octavio 
 wrote:

> ---...---
> Este email es para el staff del proyecto Guix.
> ___
> Cualquier otra persona no autorizada por mi Octavio Alfredo Toscano Flores 
> que lo esté leyendo le informo que está violando mi intimidad y privacidad.
> -----
>
> Hello everyone.
>
> I was reading.
>
> The specific URL: guix.gnu.org/es/
> I found the specific text:
> "Guix es una avanzada distribución del sistema operativo GNU desarrollada por 
> el proyecto GNU"
>
> Also.
>
> The specific URL:
> guix.gnu.org/en/
> I found the specific text:
> "Guix is an advanced distribution of the GNU operating system"
>
> I was wondering.
>
> Why Guix is an advanced distribution of the GNU operating system?
>
> Muchas gracias.
> Saludos cordiales.
>
> Ing. Octavio Toscano.

Advanced distribution

[Engineer Octavio]

---...---
Este email es para el staff del proyecto Guix.
___
Cualquier otra persona no autorizada por mi Octavio Alfredo Toscano Flores
que lo esté leyendo le informo que está violando mi intimidad y privacidad.
-----

Hello everyone.

I was reading.

The specific URL: guix.gnu.org/es/
I found the specific text:
"Guix es una avanzada distribución del sistema operativo GNU desarrollada
por el proyecto GNU"

Also.

The specific URL:
guix.gnu.org/en/
I found the specific text:
"Guix is an advanced distribution of the GNU operating system"

I was wondering.

Why Guix is an advanced distribution of the GNU operating system?

Muchas gracias.
Saludos cordiales.

Ing. Octavio Toscano.

Re: advanced?

[zimoun]

Hi,

On Mon, 28 Nov 2022 at 15:44, Simon Josefsson via "Development of GNU Guix and 
the GNU System distribution."  wrote:

> Yes, that makes sense.  I'm not the best person to summarize it, but
> starting pointers if someone wants to take it further:

Well, it is somehow part of,

* Transactional upgrades and roll-backs
* Reproducible build environments

I would also add Inferiors and Time-machine, especially working in
tandem with Software Heritage.  AFAIU, it is unique to be able to jump
to (almost) any point back in time and just rebuild (or almost), with
one command-line, whatever the state of the world (or almost).  It
pushes far beyond features such as https://snapshot.debian.org/ IMHO.

Cheers,
simon

Re: advanced?

[Denis 'GNUtoo' Carikli]

On Sun, 27 Nov 2022 10:35:13 -0800
Vagrant Cascadian  wrote:
> It also makes me wonder if "advanced" will stand the test of
> time. Someday Guix-style systems might just be status quo, and thus no
> longer advanced. Guix of course will likely evolve over time... maybe
> it will still hold qualities worthy of being called "advanced", [...]
Something interesting would be to convey what users need to know or
learn for using Guix. In "1.1 Managing Software the Guix Way" we
already have hints that it might require to know the command line and
scheme. Though maybe it could be clarified for less technical users.

For instance it "provides" [a command line interface], but that is not
clear that it's the only way to interact with some of Guix features.

If we compare Guix with other FSDG compliant distributions:
- Trisquel is usable by users that don't know the command
  line but less technical users might need a bit of help for upgrading
  from a version to another (in install parties for instance). Sometimes
  they just need somebody to be there just in case something goes wrong
  though.
- Parabola x86_64 can probably be used by users without command line
  knowledge (for a desktop/laptop usage) but the boot sometimes break,
  so less technical users also need to plan ahead and know how to
  reinstall it if needed (that could be done by having a separate home
  for instance). A server usage does require to know the command line
  and also to know how to edit configuration files (like Apache
  configuration file).
- Once installed, LibreCMC (and OpenWRT) are probably also relatively
  easy to configure for people that know what an IP address is, what is
  DHCP, what is an SSID, etc. Guix has the potential to be similar.
- Freedombox (available in PureOS, Debian, etc) looks way easier but it
  is also way less configurable.

Guix has the potential to have the same kind of balance between
easiness and empowerment/configurability than LibreCMC (if
graphical interfaces are written).

Making the current status more clear can probably help users. On my side
I've already taken that into account on the documentation I wrote on
FSDG compliant distributions on the Libreplanet wiki, but I'm not sure
how to improve the text in that manual section, or how to promote more
that information.

Denis.


pgptBDqM3kKVK.pgp
Description: OpenPGP digital signature

Re: advanced?

[Development of GNU Guix and the GNU System distribution.]

Thanks Liliana, zimoun, Ryan and Vagrant for feedback!

Ludovic Courtès  writes:

> Or if we do want to explain more, then perhaps we need a list of
> features that would also include things like Docker/VM image generation,
> declarative home environments, etc.  But that’s broader topic.

Yes, that makes sense.  I'm not the best person to summarize it, but
starting pointers if someone wants to take it further:

* Dedication to free software goals and the GNU community

* Shepherd init system written in Guile

* Declarative stateless system configurations

* Transactional upgrades and roll-backs

* Reproducible build environments

* Designed towards bootstrappable builds

Maybe this fits better directly in the Introduction section of the
manual?  https://guix.gnu.org/manual/en/html_node/Introduction.html

> PS: For the record, the phrase “advanced distribution of the GNU system”
> was coined by RMS at a time where he insisted that this thing cannot
> be called “the GNU system”.  All this makes little sense, even less
> so today, but if you’re curious you may enjoy Andreas’ entertaining
> talk: https://10years.guix.gnu.org/video/ten-years-of-failures/

Ah, thanks, the wording of that paragraph is more understandable now!  I
can see how that wording came about, and also how it clarify compared to
the GNU system.  I think this knowledge was the missing piece I didn't
have.  As an introduction to what Guix is for someone without earlier
understanding of GNU etc, I still believe that the word 'advanced' does
not contribute though.

/Simon


signature.asc
Description: PGP signature

Re: advanced?

[Ludovic Courtès]

Hello!

Vagrant Cascadian  skribis:

> On 2022-11-26, Simon Josefsson via "Development of GNU Guix and the GNU 
> System distribution." wrote:
>> I find use of the term 'advanced' wrt Guix confusing and even mildly
>> excluding, even though it is wide-spread.  What is advanced about Guix?
>> Can I use it even if I'm not an advanced user?  What do others think?
>> Is there some historical background for this description of Guix?
>
> Thanks for bringing this up!
>
> It does seem consistent with the guix manual section on package synopsis
> and descriptions:
>
>   https://guix.gnu.org/en/manual/devel/en/guix.html#Synopses-and-Descriptions

Indeed.  :-)  I’m fine with the patch Simon submitted and agree with the
rationale.

[...]

>> If we want to use the term, I think it would be better to rephrase
>> things as 'Guix supports advanced features such as X, Y and Z' if we
>> really want to drive home that we are advanced.
>
> This works for me... describe *why* it is advanced rather than just
> proclaiming it.

The second and third points (dependable and hackable) give an idea of
what makes it advanced, so maybe we don’t need to add much?

Or if we do want to explain more, then perhaps we need a list of
features that would also include things like Docker/VM image generation,
declarative home environments, etc.  But that’s broader topic.

Thanks,
Ludo’.

PS: For the record, the phrase “advanced distribution of the GNU system”
was coined by RMS at a time where he insisted that this thing cannot
be called “the GNU system”.  All this makes little sense, even less
so today, but if you’re curious you may enjoy Andreas’ entertaining
talk: https://10years.guix.gnu.org/video/ten-years-of-failures/

Re: advanced?

[Vagrant Cascadian]

On 2022-11-26, Simon Josefsson via "Development of GNU Guix and the GNU System 
distribution." wrote:
> I find use of the term 'advanced' wrt Guix confusing and even mildly
> excluding, even though it is wide-spread.  What is advanced about Guix?
> Can I use it even if I'm not an advanced user?  What do others think?
> Is there some historical background for this description of Guix?

Thanks for bringing this up!

It does seem consistent with the guix manual section on package synopsis
and descriptions:

  https://guix.gnu.org/en/manual/devel/en/guix.html#Synopses-and-Descriptions

  Please avoid marketing phrases such as “world-leading”,
  “industrial-strength”, and “next-generation”, and avoid superlatives
  like “the most advanced”—they are not helpful to users looking for a
  package and may even sound suspicious. Instead, try to be factual,
  mentioning use cases and features.

I'm just not sure what stating "advanced" up-front really adds or
improves a brief statement about what guix is...

Reading through your patch, it just seems like an extra word thrown in
hoping for the positive connotations and possibly dragging in some
negative ones (e.g. elitist, not for everybody).

It also makes me wonder if "advanced" will stand the test of
time. Someday Guix-style systems might just be status quo, and thus no
longer advanced. Guix of course will likely evolve over time... maybe it
will still hold qualities worthy of being called "advanced", maybe not.

Reminds me of when I used to work at a computer re-use and recycling
organization, and we would routingly get computers with stickers on them
proclaiming "Blazing fast 400MHz processor" and the likes. Marketing
phrases quickly loose context.


> If we want to use the term, I think it would be better to rephrase
> things as 'Guix supports advanced features such as X, Y and Z' if we
> really want to drive home that we are advanced.

This works for me... describe *why* it is advanced rather than just
proclaiming it.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: advanced?

[Ryan Prior]

On Saturday, November 26th, 2022 at 9:47 PM, Simon Josefsson via "Development 
of GNU Guix and the GNU System distribution."  wrote:

> I find use of the term 'advanced' wrt Guix confusing and even mildly 
> excluding, even though it is wide-spread. [...] Can I use it even if I'm not 
> an advanced user? What do others think?


I'll offer a perspective as a native English speaker who reads and writes a lot 
about software.

tl;dr: the word "advanced" can be offputting and give an exclusive vibe, 
because of the ways it is typically used in idiomatic English prose written 
about software. But to a small fraction of people it is the opposite, it is 
welcoming and inclusive.

In software marketing, if a product or solution is described as "advanced," 
that typically communicates that it considers and caters to demanding use-cases 
which are beyond what most people face. If you want to reach people who are 
struggling with distributed system uptime, describing an "advanced clustering 
solution" could be a good way to connect, for example.

In software documentation and configuration, "advanced" is used as shorthand 
for "this is safe to ignore." Less-technical users feel reassured that they can 
skip an "advanced" section entirely, never read it, and not miss anything that 
would be relevant to them. A technical user seeing "advanced" knows that this 
might be interesting to them, but maybe not on the first read-through before 
they are familiar with high level concepts; it's safe to skip for now. Many 
applications have a section of their settings menu labeled "advanced" - this 
too is a shorthand for "safe to ignore." Many users will never even glance at 
the advanced settings of any application they use, and even power users will 
often wait until they have some experience with an application before diving 
into advanced settings.

To a small subset of hackers and techies, "advanced" is a welcome-word: it 
says, this is something pithy that we included for those who dare to demand 
flexibility and utility. For those users, an "advanced" software product is 
likely to be more interesting even if it's hard to use, and they dive into 
"advanced" configurations immediately in case there's interesting insights 
about software internals and capabilities in there. This is wholesome and 
commendable behavior, but IMO folks who behave this way should consider that 
they are a fraction of one percent and their experience of software is in many 
ways unrelatable to that of their comrades.

Re: advanced?

[zimoun]

Hi Simon,

On Sat, 26 Nov 2022 at 22:47, Simon Josefsson via "Development of GNU Guix and 
the GNU System distribution."  wrote:

> The patch is meant as food for thought as I'm sure I'm missing
> something, and it may be a language/cultural thing.

I am not native-english.  The dictionary says:

 1. An advanced system, method, or design is modern and has been developed
from an earlier version of the same thing.

 2. Something that is at an advanced stage or level is at a late stage of
development.

 3. An advanced student has already learned the basic facts of a subject
and is doing more difficult work. An advanced course of study is
designed for such students.  

IIUC, you are understanding “advanced” as #3 when it can also be
understood as #1 or #2.

Well, I do not have an opinion about the term “advanced” for Guix.  For
sure, it seems worth to underline that the concepts are not as usual
distro. :-)


Cheers,
simon

Re: advanced?

[Liliana Marie Prikler]

Hi Simon,

Am Samstag, dem 26.11.2022 um 22:47 +0100 schrieb Simon Josefsson via
Development of GNU Guix and the GNU System distribution.:
> Hi
> 
> I find use of the term 'advanced' wrt Guix confusing and even mildly
> excluding, even though it is wide-spread.  What is advanced about
> Guix?
Guix is advanced – or perhaps more accurately revolutionary – in a
cultural sense.  Other well-known GNU distributions take an already
established non-GNU distribution and remove software that do not
respect user freedom.  See [1]: 5 out of 8 GNU distributions are "based
on" other distros.

Guix does not inherit any of that, it has its own way of describing
packages as well as its own way of describing anything else that goes
into an operating system.  You may see similarity with Nix mentioned
from time to time on social media and elsewhere, but that really is
just a shallow lake under a huge castle.

> Can I use it even if I'm not an advanced user?  
Probably maybe.  Whether or not you're an advanced user doesn't really
have any weight in there, but to get the "true" experience of a free
operating system, you do need the right hardware.  "Advanced" users are
more likely to pick their parts correctly given the existing
recommendations, but that's knowledge that can be achieved through
trial and error.

> What do others think?
About you using Guix user even though your not a "power" user?  Not
much, really.  We'll be happy to solve problems if they arise both in
the IRC and on help-g...@gnu.org

As for people, who do not use or maybe haven't even heard of Guix, I do
think a fair amount of them would treat you as though you announced
that you're vegan at some barbecue if you told them that you're going
to use or are already using a completely free, declarative operating
system.  

> Is there some historical background for this description of Guix?
Guix is historically very avant-garde, also with its reduction of the
bootstrap seed based on Mes, or its bootstrapping of the Java ecosystem
and other parts of the software world.

> How about the attached patch to merely drop this term in some place?
I don't think dropping it would be a good idea.  For one, it does have
an actual meaning as described above, but also, it evokes a feeling
that Guix is something capable of doing "real world user stuff" and not
just a neat toy.  And we definitely want to be in the former category.

> I would happily agree that Guix supports and encourages advanced
> features.  But that is not exactly the same as saying Guix itself is
> advanced.  If we want to use the term, I think it would be better to
> rephrase things as 'Guix supports advanced features such as X, Y and
> Z' if we really want to drive home that we are advanced.
It's not that Guix "supports" advanced features.  Guix itself is an
advanced way of doing the things Guix does, namely package management,
system management, container/vm spawning, ...

> I would prefer to use a basic or even simple operating system, and
> I'd like to think Guix can be that for me, but maybe everyone has
> different preferences, so maybe it doesn't really have to say
> anything.
Guix is simple, but basic only in that it breaks basically every
existing tradition before it.  Depending on your actual needs – e.g.
small disk footprint – there might be some that we are not going to
satisfy, but you're welcome to try and see if it fits regardless.


Hope that helps.

Cheers,
Liliana

[1] https://www.gnu.org/distros/free-distros.html

advanced?

[Development of GNU Guix and the GNU System distribution.]

Hi

I find use of the term 'advanced' wrt Guix confusing and even mildly
excluding, even though it is wide-spread.  What is advanced about Guix?
Can I use it even if I'm not an advanced user?  What do others think?
Is there some historical background for this description of Guix?

How about the attached patch to merely drop this term in some place?

I would happily agree that Guix supports and encourages advanced
features.  But that is not exactly the same as saying Guix itself is
advanced.  If we want to use the term, I think it would be better to
rephrase things as 'Guix supports advanced features such as X, Y and Z'
if we really want to drive home that we are advanced.

I would prefer to use a basic or even simple operating system, and I'd
like to think Guix can be that for me, but maybe everyone has different
preferences, so maybe it doesn't really have to say anything.

(Or we could re-claim a variant of Debian's old slogan 'Guix - a free
universal operating system' but that may be controversial...)

The patch is meant as food for thought as I'm sure I'm missing
something, and it may be a language/cultural thing.

/Simon
From aac8f6d1fb382b9f9120b7cd51dc80e8ef07cc03 Mon Sep 17 00:00:00 2001
From: Simon Josefsson 
Date: Sat, 26 Nov 2022 22:35:15 +0100
Subject: [PATCH] website: Reduce use of 'advanced' term.

---
 website/apps/base/templates/about.scm | 4 ++--
 website/apps/base/templates/home.scm  | 6 +++---
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/website/apps/base/templates/about.scm b/website/apps/base/templates/about.scm
index 9e2d252..1cf2f65 100644
--- a/website/apps/base/templates/about.scm
+++ b/website/apps/base/templates/about.scm
@@ -16,7 +16,7 @@
   (theme
#:title (C_ "webpage title" '("About"))
#:description
-   (G_ "Guix is an advanced distribution of the GNU operating system.
+   (G_ "Guix is a distribution of the GNU operating system.
 Guix is technology that respects the freedom of computer users.
 You are free to run the system for any purpose, study how it
 works, improve it, and share it with the whole world.")
@@ -45,7 +45,7 @@ system|GNU Hurd|GNU Guix package manager") #\|)
 
   ,(G_
     `(p
-  "Guix System is an advanced distribution of the "
+  "Guix System is a distribution of the "
   ,(G_ `(a (@ (href ,(gnu-url))) "GNU operating system"))
   ".  It uses the "
   ,(G_ `(a (@ (href ,(gnu-url "software/linux-libre"))) "Linux-libre"))
diff --git a/website/apps/base/templates/home.scm b/website/apps/base/templates/home.scm
index 7b72da8..8a0bccd 100644
--- a/website/apps/base/templates/home.scm
+++ b/website/apps/base/templates/home.scm
@@ -18,9 +18,9 @@
   "Return the Home page in SHTML using the data in CONTEXT."
   (theme
#:title (C_ "webpage title"
-   '("GNU's advanced distro and transactional package manager"))
+   '("GNU Guix transactional package manager and distribution"))
#:description
-   (G_ "Guix is an advanced distribution of the GNU operating system.
+   (G_ "Guix is a distribution of the GNU operating system.
Guix is technology that respects the freedom of computer users.
You are free to run the system for any purpose, study how it
works, improve it, and share it with the whole world.")
@@ -45,7 +45,7 @@ management|Reproducibility") #\|)
,(G_
  `(li
,(G_ `(b "Liberating."))
-   " Guix is an advanced distribution of the "
+   " Guix is a distribution of the "
,(G_ (link-yellow
  #:label "GNU operating system"
  #:url (gnu-url "gnu/about-gnu.html")))
-- 
2.30.2



signature.asc
Description: PGP signature

Re: Advanced network configuration

[Ludovic Courtès]

Hi,

Alexey Abramov  skribis:

>> I would do that by having ‘networking’ depend on ‘firewall’ (say).
>>
>> Does that make sense?
>>
>> It’d be interesting to see whether we need something beyond this.
>
> But what if I just want to stop a firewall? Won't that trigger
> network to restart in that case?

Yes it will, you’re right.

>>> Applications have to be able to gracefully shutdown their network
>>> connections.  Is it the case right now, I don't know?
>>
>> What do you mean?
>
> If you run a simple VM with openssh, login there via ssh client and run
> 'herd power-off root' from the QEMU serial console, the VM will be
> shutdown, but your ssh connection won't know anything about it and the
> socket will be left in a ESTABLISHED state! By typing there system will
> keep trying to send data over there wire (Send-Q).
>
> root@delta ~# ss -ta -A all '( dport = :ssh )' dst 192.168.10.100/24
> Netid State Recv-Q Send-Q 
> Local Address:PortPeer Address:Port   
>  Process 
> tcp   ESTAB 0  288
>  192.168.10.1:48106 192.168.10.100:ssh
>  
>
> There is no such a problem with shutdown command by the way. 

Hmm I see.

So I guess I don’t know what the correct solution would be, but it’d be
great to see what ideas you have and how systemd approaches these
issues.  Then, if needed, we can adjust the service model in the
Shepherd.

Thanks,
Ludo’.

Re: Advanced network configuration

[Alexey Abramov]

Hi,

Ludovic Courtès  writes:

> Hi!
>
> Alexey Abramov  skribis:
>
> [...]
>

[...]

>
> I would do that by having ‘networking’ depend on ‘firewall’ (say).
>
> Does that make sense?
>
> It’d be interesting to see whether we need something beyond this.

But what if I just want to stop a firewall? Won't that trigger
network to restart in that case?

>> Applications have to be able to gracefully shutdown their network
>> connections.  Is it the case right now, I don't know?
>
> What do you mean?

If you run a simple VM with openssh, login there via ssh client and run
'herd power-off root' from the QEMU serial console, the VM will be
shutdown, but your ssh connection won't know anything about it and the
socket will be left in a ESTABLISHED state! By typing there system will
keep trying to send data over there wire (Send-Q).

--8<---cut here---start->8---
root@delta ~# ss -ta -A all '( dport = :ssh )' dst 192.168.10.100/24
Netid State Recv-Q Send-Q   
  Local Address:PortPeer Address:Port   
 Process 
tcp   ESTAB 0  288  
   192.168.10.1:48106 192.168.10.100:ssh
 
--8<---cut here---end--->8---

There is no such a problem with shutdown command by the way. 

>> I am checking (shepherd services) where `shutdown-services' defined, and
>> seems like it just walks across %services hash table. Am I missing
>> something?
>
> Correct, there’s nothing fancy going on there.
>
> Thanks,
> Ludo’.

-- 
Alexey

Re: Advanced network configuration

[Julien Lepiller]

Le 10 octobre 2022 17:17:16 GMT+02:00, "Ludovic Courtès"  a écrit 
:
>Hi!
>
>Alexey Abramov  skribis:
>
>[...]
>
>>> I’m not sure.  IIUC, a “networking target” here could translate to a
>>> Shepherd service that depends on all the relevant DHCP and static
>>> networking services.  The question the becomes how to express that
>>> grouping conveniently.
>>
>> Yes, I also would like to point out that their must be a way to
>> establish a firewall, for example, *before* any network interface is up
>> (After=network-pre.target in systemd [1]). And the same thing during the
>> shutdown procedure (Before=network-pre.target in systemd).
>
>I would do that by having ‘networking’ depend on ‘firewall’ (say).
>
>Does that make sense?

Wouldn't there be an issue if firewall rules reference interfaces created by 
the networking service?

>
>It’d be interesting to see whether we need something beyond this.
>

Re: Advanced network configuration

[Ludovic Courtès]

Hi!

Alexey Abramov  skribis:

[...]

>> I’m not sure.  IIUC, a “networking target” here could translate to a
>> Shepherd service that depends on all the relevant DHCP and static
>> networking services.  The question the becomes how to express that
>> grouping conveniently.
>
> Yes, I also would like to point out that their must be a way to
> establish a firewall, for example, *before* any network interface is up
> (After=network-pre.target in systemd [1]). And the same thing during the
> shutdown procedure (Before=network-pre.target in systemd).

I would do that by having ‘networking’ depend on ‘firewall’ (say).

Does that make sense?

It’d be interesting to see whether we need something beyond this.

> Applications have to be able to gracefully shutdown their network
> connections.  Is it the case right now, I don't know?

What do you mean?

> I am checking (shepherd services) where `shutdown-services' defined, and
> seems like it just walks across %services hash table. Am I missing
> something?

Correct, there’s nothing fancy going on there.

Thanks,
Ludo’.

Re: Advanced network configuration

[Alexey Abramov]

Hi Julien,

I sent patches to guile-netlink [1].


Footnotes:
[1]  https://issues.guix.gnu.org/58382

-- 
Alexey

Re: Advanced network configuration

[Julien Lepiller]

I guess using debbugs would give other people a chance to have a look at your 
patches and comment, but I'm the only one who can push anyway. If you decide to 
use debbugs, make sure to CC me too.

Le 6 octobre 2022 15:11:30 GMT+02:00, Alexey Abramov  a 
écrit :
>Hi Ludo, Julien
>
>Ludovic Courtès  writes:
>
>> Hi Alexey,
>>
>> (Cc: Julien, author of Guile-Netlink.)
>>
>
>[...]
>
>>
>> I’m sure your improvements to Guile-Netlink would be welcome.
>
>> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
>> expressive as the underlying netlink interface, but clearly we focused
>> on the most common use cases.
>>
>> If you can think of how you’d like to represent these setups in
>> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
>> snippet you showed?), we (or you :-)) can try and implement it.
>
>Yeah, that was my intention =). @Julien Could you tell me how can I
>collaborate? Shall I send patches to you directly or maybe debbugs, or
>guix-patches?
>
>>> 2. Having a router with Guix at home. I have to run multiple services
>>> that provision 'networking' which is not allowed right now. The DHCP
>>> client service is greedy right now and binds to all available
>>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>>> dhcp-client and static configuration at the same time anyway.
>>
>> OK, we could allow users to change the Shepherd service name used by the
>> DHCP client then.
>
>That would indeed help for now. I can prepare yet another patch for
>that.
>
>[...]
>
>> I’m not sure.  IIUC, a “networking target” here could translate to a
>> Shepherd service that depends on all the relevant DHCP and static
>> networking services.  The question the becomes how to express that
>> grouping conveniently.
>
>Yes, I also would like to point out that their must be a way to
>establish a firewall, for example, *before* any network interface is up
>(After=network-pre.target in systemd [1]). And the same thing during the
>shutdown procedure (Before=network-pre.target in systemd).  Applications
>have to be able to gracefully shutdown their network connections.  Is it
>the case right now, I don't know?
>
>I am checking (shepherd services) where `shutdown-services' defined, and
>seems like it just walks across %services hash table. Am I missing
>something?
>
>Footnotes:
>[1]  
>https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target
>
>-- 
>Alexey

Re: Advanced network configuration

[Alexey Abramov]

Hi Ludo, Julien

Ludovic Courtès  writes:

> Hi Alexey,
>
> (Cc: Julien, author of Guile-Netlink.)
>

[...]

>
> I’m sure your improvements to Guile-Netlink would be welcome.

> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
> expressive as the underlying netlink interface, but clearly we focused
> on the most common use cases.
>
> If you can think of how you’d like to represent these setups in
> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
> snippet you showed?), we (or you :-)) can try and implement it.

Yeah, that was my intention =). @Julien Could you tell me how can I
collaborate? Shall I send patches to you directly or maybe debbugs, or
guix-patches?

>> 2. Having a router with Guix at home. I have to run multiple services
>> that provision 'networking' which is not allowed right now. The DHCP
>> client service is greedy right now and binds to all available
>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>> dhcp-client and static configuration at the same time anyway.
>
> OK, we could allow users to change the Shepherd service name used by the
> DHCP client then.

That would indeed help for now. I can prepare yet another patch for
that.

[...]

> I’m not sure.  IIUC, a “networking target” here could translate to a
> Shepherd service that depends on all the relevant DHCP and static
> networking services.  The question the becomes how to express that
> grouping conveniently.

Yes, I also would like to point out that their must be a way to
establish a firewall, for example, *before* any network interface is up
(After=network-pre.target in systemd [1]). And the same thing during the
shutdown procedure (Before=network-pre.target in systemd).  Applications
have to be able to gracefully shutdown their network connections.  Is it
the case right now, I don't know?

I am checking (shepherd services) where `shutdown-services' defined, and
seems like it just walks across %services hash table. Am I missing
something?

Footnotes:
[1]  
https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target

-- 
Alexey

Re: Advanced network configuration

[Ludovic Courtès]

Hi Alexey,

(Cc: Julien, author of Guile-Netlink.)

Alexey Abramov  skribis:

> 1. I have servers with multiple network cards. Let's say, the management
> one and production. both network cards have multiple interfaces, which I
> need to bond into 803.1ad mode. Management bond has a native vlan where
> DHCP should be provided AND another vlan for the management traffic. The
> production interface (after bonding) has another set of VLANs for a
> different type of traffic as well. Currently, there is no way to define
> it without writing a custom service with the iproute2 utility.
>
> I was hacking on guile-netlink and did some progress with setting
> bonds. However current static configuration won't let me build it
> anyway. I have to not only 'link-add' and create a bond, but also amend
> existing interfaces by 'link-set' them as a slave to my newly created
> bond master for example.

I’m sure your improvements to Guile-Netlink would be welcome.

Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
expressive as the underlying netlink interface, but clearly we focused
on the most common use cases.

If you can think of how you’d like to represent these setups in
‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
snippet you showed?), we (or you :-)) can try and implement it.

> 2. Having a router with Guix at home. I have to run multiple services
> that provision 'networking' which is not allowed right now. The DHCP
> client service is greedy right now and binds to all available
> interfaces. I sent a [1] patch to solve this. However, I cannot define
> dhcp-client and static configuration at the same time anyway.

OK, we could allow users to change the Shepherd service name used by the
DHCP client then.

> These two examples lead me to the following questions:
>
> 1) To configure the network, I have been happily using netplan app
> (debain/ubuntu world). The specification they use can be found here
> [2]. Here is an example:

[...]

> From the example above, you can see that I rename interfaces (i do know
> that udev can do it, but it is easier to do it here). I build bonds and
> VLANs with their IPs. I also set routes and default gateway. Would
> someone be interested in that? I don't see any problems in implementing
> this with guile-netlink. This is basically my proposal on how we can
> improve our static network configuration. What do you think?

I’m all for it!  This should be done in a way that preserves
compatibility with the existing  record as much as
possible.

> 2) In Systemd world there is a notion of 'target' which is used for
> *grouping* units and *synchronization* points during the start-up.
>
> At this moment *I* need to have dhcp-client and static
> configuration. Seems like grouping is a key here. Were there any
> ideas/attempts to make it happen? Or what should I do? 

I’m not sure.  IIUC, a “networking target” here could translate to a
Shepherd service that depends on all the relevant DHCP and static
networking services.  The question the becomes how to express that
grouping conveniently.

Thanks,
Ludo’.

Re: Advanced network configuration

[Alexey Abramov]

Hi Ryan,

Ryan Sundberg  writes:

> Hi Alexy, are you running netplan under Guix? I would find it useful
> to have a service using a netplan config, bypassing the guile-based
> networking service for such advanced use cases as you mention.

No, I have never tried to do this actually. Netplan does support
NetworkManager as a backend. Maybe it is possible.

However, I would love to use a regular Guix service for that. 
guile-netlink with some small changes [1], can be used to build such a
configuration. 

I haven't implemented all the bond features, yet. Right now, I am
looking for ways to implement tests.

Footnotes:
[1]  https://gitlab.com/Levenson/guile-netlink/-/tree/wip-bonding

-- 
Alexey

Re: Advanced network configuration

[Ryan Sundberg]

Hi Alexy, are you running netplan under Guix? I would find it useful to have a 
service using a netplan config, bypassing the guile-based networking service 
for such advanced use cases as you mention.


Sincerely,

Ryan Sundberg
Principal Software Engineer
Arctype Corporation



 Original Message 
From: Alexey Abramov 
Sent: October 3, 2022 11:23:11 AM PDT
To: guix-devel@gnu.org
Subject: Advanced network configuration

Hi Guix,

I have a couple of questions regarding our current network configuration
in Guix. With the latest changes to the static configuration, live is
indeed, became easier, but I have to say that it is still insufficient
to define a complete network configuration.

Please check out the following (my) use cases:

1. I have servers with multiple network cards. Let's say, the management
one and production. both network cards have multiple interfaces, which I
need to bond into 803.1ad mode. Management bond has a native vlan where
DHCP should be provided AND another vlan for the management traffic. The
production interface (after bonding) has another set of VLANs for a
different type of traffic as well. Currently, there is no way to define
it without writing a custom service with the iproute2 utility.

I was hacking on guile-netlink and did some progress with setting
bonds. However current static configuration won't let me build it
anyway. I have to not only 'link-add' and create a bond, but also amend
existing interfaces by 'link-set' them as a slave to my newly created
bond master for example.

2. Having a router with Guix at home. I have to run multiple services
that provision 'networking' which is not allowed right now. The DHCP
client service is greedy right now and binds to all available
interfaces. I sent a [1] patch to solve this. However, I cannot define
dhcp-client and static configuration at the same time anyway.

These two examples lead me to the following questions:

1) To configure the network, I have been happily using netplan app
(debain/ubuntu world). The specification they use can be found here
[2]. Here is an example:

--8<---cut here---start->8---
network:
  bonds:
bond0:
  interfaces:
- eno0
- eno1
  parameters:
mode: 802.3ad
bond2:
  interfaces:
- enp0
- enp1
  parameters:
mode: 802.3ad
  ethernets:
eno0:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno0
eno1:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno1
eno2:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno2
eno3:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno3
enp0:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: enp0
enp1:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: enp1
  renderer: networkd
  version: 2
  vlans:
bond1.1055:
  addresses:
- 10.0.0.1/24
  routes:
- to: XX.XX.XX.XX/21
  via: 10.x.x.x
  gateway4: 10.0.0.1
  id: 1055
  link: bond1
bond2.2055:
  addresses:
- 10.8.0.1/24
  id: 2055
  link: bond2
bond2.3055:
  addresses:
- 192.168.0.0/16
  id: 3055
  link: bond2
--8<---cut here---end--->8---

From the example above, you can see that I rename interfaces (i do know
that udev can do it, but it is easier to do it here). I build bonds and
VLANs with their IPs. I also set routes and default gateway. Would
someone be interested in that? I don't see any problems in implementing
this with guile-netlink. This is basically my proposal on how we can
improve our static network configuration. What do you think?

2) In Systemd world there is a notion of 'target' which is used for
*grouping* units and *synchronization* points during the start-up.

At this moment *I* need to have dhcp-client and static
configuration. Seems like grouping is a key here. Were there any
ideas/attempts to make it happen? Or what should I do? 

Footnotes:
[1]  https://issues.guix.gnu.org/58223

[2]  https://netplan.io/reference

-- 
Alexey

Advanced network configuration

[Alexey Abramov]

Hi Guix,

I have a couple of questions regarding our current network configuration
in Guix. With the latest changes to the static configuration, live is
indeed, became easier, but I have to say that it is still insufficient
to define a complete network configuration.

Please check out the following (my) use cases:

1. I have servers with multiple network cards. Let's say, the management
one and production. both network cards have multiple interfaces, which I
need to bond into 803.1ad mode. Management bond has a native vlan where
DHCP should be provided AND another vlan for the management traffic. The
production interface (after bonding) has another set of VLANs for a
different type of traffic as well. Currently, there is no way to define
it without writing a custom service with the iproute2 utility.

I was hacking on guile-netlink and did some progress with setting
bonds. However current static configuration won't let me build it
anyway. I have to not only 'link-add' and create a bond, but also amend
existing interfaces by 'link-set' them as a slave to my newly created
bond master for example.

2. Having a router with Guix at home. I have to run multiple services
that provision 'networking' which is not allowed right now. The DHCP
client service is greedy right now and binds to all available
interfaces. I sent a [1] patch to solve this. However, I cannot define
dhcp-client and static configuration at the same time anyway.

These two examples lead me to the following questions:

1) To configure the network, I have been happily using netplan app
(debain/ubuntu world). The specification they use can be found here
[2]. Here is an example:

--8<---cut here---start->8---
network:
  bonds:
bond0:
  interfaces:
- eno0
- eno1
  parameters:
mode: 802.3ad
bond2:
  interfaces:
- enp0
- enp1
  parameters:
mode: 802.3ad
  ethernets:
eno0:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno0
eno1:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno1
eno2:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno2
eno3:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: eno3
enp0:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: enp0
enp1:
  match:
macaddress: xx:xx:xx:xx:xx:xx
  set-name: enp1
  renderer: networkd
  version: 2
  vlans:
bond1.1055:
  addresses:
- 10.0.0.1/24
  routes:
- to: XX.XX.XX.XX/21
  via: 10.x.x.x
  gateway4: 10.0.0.1
  id: 1055
  link: bond1
bond2.2055:
  addresses:
- 10.8.0.1/24
  id: 2055
  link: bond2
bond2.3055:
  addresses:
- 192.168.0.0/16
  id: 3055
  link: bond2
--8<---cut here---end--->8---

>From the example above, you can see that I rename interfaces (i do know
that udev can do it, but it is easier to do it here). I build bonds and
VLANs with their IPs. I also set routes and default gateway. Would
someone be interested in that? I don't see any problems in implementing
this with guile-netlink. This is basically my proposal on how we can
improve our static network configuration. What do you think?

2) In Systemd world there is a notion of 'target' which is used for
*grouping* units and *synchronization* points during the start-up.

At this moment *I* need to have dhcp-client and static
configuration. Seems like grouping is a key here. Were there any
ideas/attempts to make it happen? Or what should I do? 

Footnotes:
[1]  https://issues.guix.gnu.org/58223

[2]  https://netplan.io/reference

-- 
Alexey

FW: Guix: A most advanced operating system

[Cook, Malcolm]

FWIW: Discussion of this article https://ambrevar.xyz/guix-advance/index.html 
occurring on hacker news now  at https://news.ycombinator.com/item?id=18902823

Re: Optionally using more advanced CPU features

[Ludovic Courtès]

Hi Ben,

Ben Woodcroft  skribis:

> Anyway, to move forward I created a repo so that package recipes can
> be modified to use a GCC that has been optimised for a particular
> architecture. I put it out there so that it is more than just a patch
> on this ML, but I'd be happy to incorporate it into Guix proper if
> that is desired.
> https://github.com/wwood/cpu-specific-guix
>
> For instance, to build DIAMOND optimised for sandybridge:
>
> GUILE_LOAD_PATH=/path/to/cpu-specific-guix:$GUILE_LOAD_PATH\ |guix
> build -e '(begin (use-modules (cpu-specific-guix) (gnu packages
> bioinformatics))\ (cpu-specific-package diamond "sandybridge"))'|

That’s a neat hack!

It’s a bit of a sledgehammer, in that we could achieve this without
rebuilding GCC, I think.  For instance, we could create a ‘gcc’ wrapper
that automatically passes “-march=foo” on the command line of the real
‘gcc’, no?

Thanks for sharing!

Ludo’.

Re: Optionally using more advanced CPU features

[Ludovic Courtès]

Hello,

Dave Love  skribis:

> Ludovic Courtès  writes:

[...]

>> But that sounds similar to IFUNC in that application code would need to
>> actually use hwcap info to select the right implementation at load time,
>> right?
>
> As far as I know, it's a loader feature.  See "Hardware capabilities" in
> ld.so(1).

Indeed.

I’ve looked at the newish libmvec along with the “Vector ABI” in the
toolchain and it’s really the kind of thing we’re looking for.

> Well, you could embark on some sort of (GCC-specific?) re-write, but it
> would be better to work on .
> I don't think there's anywhere you can just attach GCC attributes, and
> certainly no magic will happen for currently-unsupported architectures.

Agreed, adjusting BLIS to do some load-time configuration seems like the
right thing.  Thanks for the reference, we’ll see how it goes.

Ludo’.

Re: Optionally using more advanced CPU features

[Dave Love]

Ludovic Courtès  writes:

>> That may be the best way to handle it, but it's not widely available,
>> and isn't possible generally (as far as I know), e.g. for Fortran code.
>> See also below.  This issue surfaced again recently in Fedora.
>
> Right.  Do you have examples of Fortran packages in mind?

Not much off-hand because, shall we say, there's a shortage of the sort
of profiling information that's necessary for system performance
engineering and procurement.  It's not in Guix, but cp2k is a (mainly)
Fortran program that is, or was, used as performance regression test for
GCC.  I only know about its profile for cases where time in MPI or fftw
is most relevant.  However, two of its kernels, ELPA, and libsmm (as
libxsmm) have low-level optimized versions for x86_64, but only Fortran
implementations for other architectures as far as I know.

Otherwise, BLAS/LAPACK for any micro-architectures that don't have
support in free optimized variants like OpenBLAS.

>> In cases that don't dispatch on cpuid (or whatever), I think the
>> relevant missing OS/tool support is SIMD-specific hwcaps in the loader.
>> Hwcaps seem to be essentially undocumented, but there is, or has been,
>> support for instruction set capabilities on some architectures, just not
>> x86_64 apparently.  (An ancient example was for missing instructions on
>> some SPARC systems which greatly affected crypto operations in ssh et
>> al.)
>
> But that sounds similar to IFUNC in that application code would need to
> actually use hwcap info to select the right implementation at load time,
> right?

As far as I know, it's a loader feature.  See "Hardware capabilities" in
ld.so(1).

> >> There’s probably scientific software out there that can benefit from
> >> using the latest SSE/AVX/whatever extension, and yet doesn’t use any of
> >> the tricks above.  When we find such a piece of software, I think we
> >> should investigate and (1) see whether it actually benefits from those
> >> ISA extensions, and (2) see whether it would be feasible to just use
> >> ‘target_clones’ or similar on the hot spots.
> >
> >> One example which has been investigated, and you can't, is BLIS.  You
>
> (Why “you can’t?”  It’s free software AFAICS on
> .)

Well, you could embark on some sort of (GCC-specific?) re-write, but it
would be better to work on .
I don't think there's anywhere you can just attach GCC attributes, and
certainly no magic will happen for currently-unsupported architectures.

>> need it for vaguely competitive avx512 linear algebra.  (OpenBLAS is
>> basically fine for previous Intel and AMD SIMD.)  See, e.g.,
>> 
>> et seq.  I don't know if there's any good reason to, but if you want
>> ATLAS you have the same issue -- along with extra issues building it.
>
> ATLAS is a problem because it does built-time ISA selection (and maybe
> profile-guided optimization?).

Yes, that's what I meant.  (I can't remember to what extent you can just
specify the architecture and build it without the parameter sweep.)

> I sympathize with the idea of having several ABI-compatible BLAS
> implementations for the reasons you give.  That somewhat conflicts with
> the idea of reproducibility, but after all we can have our cake and eat
> it too: the user can decide to have LD_LIBRARY_PATH point to an
> alternate ABI-compatible BLAS, or they can keep using the one that
> appears in RUNPATH.
>
> Thoughts?

Right, about the cake -- as with other packaging systems -- and
LD_LIBRARY_PATH/LD_PRELOAD are important for debugging and measurement
anyway.  [I know too much about computing and experimental science to
believe in reproducibility as it's normally talked about, though
facilities for reproducible builds and environment components are good.]

Re: Optionally using more advanced CPU features

[Ludovic Courtès]

Hi Dave,

Dave Love  skribis:

> ludovic.cour...@inria.fr (Ludovic Courtès) writes:

[...]

>> To some extent, I think this is a compiler/OS/upstream issue.  By that I
>> mean that the best way to achieve use of extra CPU features is by using
>> the “IFUNC” feature of GNU ld.so, which is what libc does (it has
>> variants of strcmp etc. tweaked for various CPU extensions like SSE, and
>> the right one gets picked up at load time.)  Software like GMP, Nettle,
>> or MPlayer also does this kind of selection at run time, but using
>> custom mechanisms.
>
> That may be the best way to handle it, but it's not widely available,
> and isn't possible generally (as far as I know), e.g. for Fortran code.
> See also below.  This issue surfaced again recently in Fedora.

Right.  Do you have examples of Fortran packages in mind?

> In cases that don't dispatch on cpuid (or whatever), I think the
> relevant missing OS/tool support is SIMD-specific hwcaps in the loader.
> Hwcaps seem to be essentially undocumented, but there is, or has been,
> support for instruction set capabilities on some architectures, just not
> x86_64 apparently.  (An ancient example was for missing instructions on
> some SPARC systems which greatly affected crypto operations in ssh et
> al.)

But that sounds similar to IFUNC in that application code would need to
actually use hwcap info to select the right implementation at load time,
right?

>> There’s probably scientific software out there that can benefit from
>> using the latest SSE/AVX/whatever extension, and yet doesn’t use any of
>> the tricks above.  When we find such a piece of software, I think we
>> should investigate and (1) see whether it actually benefits from those
>> ISA extensions, and (2) see whether it would be feasible to just use
>> ‘target_clones’ or similar on the hot spots.
>
> One example which has been investigated, and you can't, is BLIS.  You

(Why “you can’t?”  It’s free software AFAICS on
.)

> need it for vaguely competitive avx512 linear algebra.  (OpenBLAS is
> basically fine for previous Intel and AMD SIMD.)  See, e.g.,
> 
> et seq.  I don't know if there's any good reason to, but if you want
> ATLAS you have the same issue -- along with extra issues building it.

ATLAS is a problem because it does built-time ISA selection (and maybe
profile-guided optimization?).

> Related, I argue, as on the Fedora list, that like BLAS (and LAPACK)
> should handled the way they are in Debian, with shared libraries built
> compatibly with the reference BLAS.  They should be selectable at run
> time, typically according to compute node type by flipping the ld.so
> search path; you should be able to substitute BLIS or a GPU
> implementation for OpenBLAS.  That likely applies in other cases, but
> I'm most familiar with the linear algebra ones.

I sympathize with the idea of having several ABI-compatible BLAS
implementations for the reasons you give.  That somewhat conflicts with
the idea of reproducibility, but after all we can have our cake and eat
it too: the user can decide to have LD_LIBRARY_PATH point to an
alternate ABI-compatible BLAS, or they can keep using the one that
appears in RUNPATH.

Thoughts?

Ludo’.

Re: Optionally using more advanced CPU features

[Pjotr Prins]

On Sat, Aug 26, 2017 at 11:39:41AM +0800, Ben Woodcroft wrote:
> I was wondering how we should go about optionally building software for
> more advanced CPU features.  Currently, we build software for the lowest
> common feature set among x86_64 CPUs.  That’s good for portability, but
> not so good for performance.
> 
>In many cases we can set the --with-arch flag when configuring GCC, so
>that packages built with that GCC are optimised for that architecture
>by default.
>We have discussed this in the past,
>([1]https://lists.gnu.org/archive/html/guix-devel/2016-10/msg5.html
>) but as you say individual packages sometimes need individual
>attention.
>Anyway, to move forward I created a repo so that package recipes can be
>modified to use a GCC that has been optimised for a particular
>architecture. I put it out there so that it is more than just a patch
>on this ML, but I'd be happy to incorporate it into Guix proper if that
>is desired.
>[2]https://github.com/wwood/cpu-specific-guix
>For instance, to build DIAMOND optimised for sandybridge:
> GUILE_LOAD_PATH=/path/to/cpu-specific-guix:$GUILE_LOAD_PATH\
>   guix build -e '(begin (use-modules (cpu-specific-guix) (gnu packages 
> bioinform
> atics))\
>   (cpu-specific-package diamond "sandybridge"))'
> 
>HTH, ben

Pretty cool. This works for leave-packages. For libraries we'll need
to have something that goes deeper into the graph. Openblas/atlas/GSL
are prime examples that would benefit a wide range of applications.

I am working on GEMMA these days and I will target supercomputing
architectures. Having math libraries that target vectorization
optimizations for gcc and LLVM would be very useful. The current
deployment strategy is 'one-offs' on the GUIX_PACKAGE_PATH.

Just as a note, it makes no sense to optimize all Guix packages. In
fact I prefer we have the non-optimized by default since that is what
everyone is using and is (arguably) well tested. We only have the 1 in
a thousand library we want to specialize/optimize aggressively.

Pj.

Re: Optionally using more advanced CPU features

[Ben Woodcroft]

Hi,


On 21/08/17 22:23, Ricardo Wurmus wrote:

Hi Guix,

I was wondering how we should go about optionally building software for
more advanced CPU features.  Currently, we build software for the lowest
common feature set among x86_64 CPUs.  That’s good for portability, but
not so good for performance.
In many cases we can set the --with-arch flag when configuring GCC, so 
that packages built with that GCC are optimised for that architecture by 
default.


We have discussed this in the past, 
(https://lists.gnu.org/archive/html/guix-devel/2016-10/msg5.html) 
but as you say individual packages sometimes need individual attention.


Anyway, to move forward I created a repo so that package recipes can be 
modified to use a GCC that has been optimised for a particular 
architecture. I put it out there so that it is more than just a patch on 
this ML, but I'd be happy to incorporate it into Guix proper if that is 
desired.

https://github.com/wwood/cpu-specific-guix

For instance, to build DIAMOND optimised for sandybridge:

GUILE_LOAD_PATH=/path/to/cpu-specific-guix:$GUILE_LOAD_PATH\ |guix build -e '(begin (use-modules (cpu-specific-guix) (gnu packages 
bioinformatics))\ (cpu-specific-package diamond "sandybridge"))'|




HTH, ben

Re: Optionally using more advanced CPU features

[Dave Love]

ludovic.cour...@inria.fr (Ludovic Courtès) writes:

> Hi,
>
> Ricardo Wurmus <rek...@elephly.net> skribis:
>
>> I was wondering how we should go about optionally building software for
>> more advanced CPU features.  Currently, we build software for the lowest
>> common feature set among x86_64 CPUs.  That’s good for portability, but
>> not so good for performance.
>>
>> Enabling CPU features often happens through configure flags, but
>> expressing support at that level in our package definitions seems bad.
>> How can we make it possible for users to build their software for
>> different CPUs?
>
> To some extent, I think this is a compiler/OS/upstream issue.  By that I
> mean that the best way to achieve use of extra CPU features is by using
> the “IFUNC” feature of GNU ld.so, which is what libc does (it has
> variants of strcmp etc. tweaked for various CPU extensions like SSE, and
> the right one gets picked up at load time.)  Software like GMP, Nettle,
> or MPlayer also does this kind of selection at run time, but using
> custom mechanisms.

That may be the best way to handle it, but it's not widely available,
and isn't possible generally (as far as I know), e.g. for Fortran code.
See also below.  This issue surfaced again recently in Fedora.

In cases that don't dispatch on cpuid (or whatever), I think the
relevant missing OS/tool support is SIMD-specific hwcaps in the loader.
Hwcaps seem to be essentially undocumented, but there is, or has been,
support for instruction set capabilities on some architectures, just not
x86_64 apparently.  (An ancient example was for missing instructions on
some SPARC systems which greatly affected crypto operations in ssh et
al.)

>> We can cross-compile for other architectures on the command line with
>> “--target” and “--system”; can we allow for compilation with special CPU
>> features across the graph with “--features”?  Build system abstractions
>> or package definitions would then be changed to recognize these features
>> and modify the corresponding flags as needed.
>
> I’ve considered this, but designing this would be tricky, and not quite
> right IMO.
>
> There’s probably scientific software out there that can benefit from
> using the latest SSE/AVX/whatever extension, and yet doesn’t use any of
> the tricks above.  When we find such a piece of software, I think we
> should investigate and (1) see whether it actually benefits from those
> ISA extensions, and (2) see whether it would be feasible to just use
> ‘target_clones’ or similar on the hot spots.

One example which has been investigated, and you can't, is BLIS.  You
need it for vaguely competitive avx512 linear algebra.  (OpenBLAS is
basically fine for previous Intel and AMD SIMD.)  See, e.g.,
<https://github.com/xianyi/OpenBLAS/issues/991#issuecomment-273631173>
et seq.  I don't know if there's any good reason to, but if you want
ATLAS you have the same issue -- along with extra issues building it.

Related, I argue, as on the Fedora list, that like BLAS (and LAPACK)
should handled the way they are in Debian, with shared libraries built
compatibly with the reference BLAS.  They should be selectable at run
time, typically according to compute node type by flipping the ld.so
search path; you should be able to substitute BLIS or a GPU
implementation for OpenBLAS.  That likely applies in other cases, but
I'm most familiar with the linear algebra ones.

[By the way, you do have to be careful with ISA-specific libraries on
heterogeneous systems if you use checkpoint-restart, as you probably
should on an HPC cluster -- you need to restart on compatible hardware.]

Re: Optionally using more advanced CPU features

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rek...@elephly.net> skribis:

> I was wondering how we should go about optionally building software for
> more advanced CPU features.  Currently, we build software for the lowest
> common feature set among x86_64 CPUs.  That’s good for portability, but
> not so good for performance.
>
> Enabling CPU features often happens through configure flags, but
> expressing support at that level in our package definitions seems bad.
> How can we make it possible for users to build their software for
> different CPUs?

To some extent, I think this is a compiler/OS/upstream issue.  By that I
mean that the best way to achieve use of extra CPU features is by using
the “IFUNC” feature of GNU ld.so, which is what libc does (it has
variants of strcmp etc. tweaked for various CPU extensions like SSE, and
the right one gets picked up at load time.)  Software like GMP, Nettle,
or MPlayer also does this kind of selection at run time, but using
custom mechanisms.

GCC now has a ‘target_clones’ function attribute, which instructs it to
generate several variants of a function and use IFUNC to pick up the
right one (info "(gcc) Common Function Attributes").  Ideally, upstream
would use this.

When upstream does that, we have portable-yet-efficient “fat” binaries,
and there’s nothing to do on our side.  :-)

> We can cross-compile for other architectures on the command line with
> “--target” and “--system”; can we allow for compilation with special CPU
> features across the graph with “--features”?  Build system abstractions
> or package definitions would then be changed to recognize these features
> and modify the corresponding flags as needed.

I’ve considered this, but designing this would be tricky, and not quite
right IMO.

There’s probably scientific software out there that can benefit from
using the latest SSE/AVX/whatever extension, and yet doesn’t use any of
the tricks above.  When we find such a piece of software, I think we
should investigate and (1) see whether it actually benefits from those
ISA extensions, and (2) see whether it would be feasible to just use
‘target_clones’ or similar on the hot spots.

If it turns out that this approach doesn’t scale or isn’t suitable, then
we can think more about what you suggest.  But before starting such an
endeavor, I would really like to get a better understanding of the
software we’re talking about and the options that we have.

WDYT?

Thanks,
Ludo’.

Optionally using more advanced CPU features

[Ricardo Wurmus]

Hi Guix,

I was wondering how we should go about optionally building software for
more advanced CPU features.  Currently, we build software for the lowest
common feature set among x86_64 CPUs.  That’s good for portability, but
not so good for performance.

Enabling CPU features often happens through configure flags, but
expressing support at that level in our package definitions seems bad.
How can we make it possible for users to build their software for
different CPUs?

We can cross-compile for other architectures on the command line with
“--target” and “--system”; can we allow for compilation with special CPU
features across the graph with “--features”?  Build system abstractions
or package definitions would then be changed to recognize these features
and modify the corresponding flags as needed.

If we had a larger build farm we could also offer substitutes for more
modern CPUs.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: [PATCH v2] gnu: wxwidgets: Enable support for their "advanced" regular expressions.

[Leo Famulari]

On Wed, Jun 01, 2016 at 07:57:11AM +0200, Danny Milosavljevic wrote:
> From: Danny Milosavljevic <danny...@scratchpost.org>
> 
> * wxwidgets: Enable support for their "advanced" regular expressions.
>  This is required by KiCad.

Thank you!

Applied as b37b854e2d6 with some more detail in the commit message.

> ---
>  gnu/packages/wxwidgets.scm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/gnu/packages/wxwidgets.scm b/gnu/packages/wxwidgets.scm
> index 2c4a26a..3231e3d 100644
> --- a/gnu/packages/wxwidgets.scm
> +++ b/gnu/packages/wxwidgets.scm
> @@ -63,7 +63,7 @@
>   `(("pkg-config" ,pkg-config)))
>  (arguments
>   '(#:configure-flags
> -   '("--with-regex=sys" "--with-libmspack"
> +   '("--with-regex" "--with-libmspack"
>   "--with-sdl"
>   "--enable-webview"
>   "--enable-webkit"

[PATCH v2] gnu: wxwidgets: Enable support for their "advanced" regular expressions.

[Danny Milosavljevic]

From: Danny Milosavljevic <danny...@scratchpost.org>

* wxwidgets: Enable support for their "advanced" regular expressions.
 This is required by KiCad.
---
 gnu/packages/wxwidgets.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/wxwidgets.scm b/gnu/packages/wxwidgets.scm
index 2c4a26a..3231e3d 100644
--- a/gnu/packages/wxwidgets.scm
+++ b/gnu/packages/wxwidgets.scm
@@ -63,7 +63,7 @@
  `(("pkg-config" ,pkg-config)))
 (arguments
  '(#:configure-flags
-   '("--with-regex=sys" "--with-libmspack"
+   '("--with-regex" "--with-libmspack"
  "--with-sdl"
  "--enable-webview"
  "--enable-webkit"

Re: [PATCH] gnu: wxwidgets: enable advanced (!) regular expressions.

[Danny Milosavljevic]

Hi,

some more docs about the so-called "advanced" regular expressions needed by 
KiCad and wxRegEx in general:

http://wxd.sourceforge.net/wxWidgets-2.5.3/docs/html/wx/wx_wxregex.html

The "advanced" regular expression features (on top of extended regular 
expressions) are:
- Escapes: \a \b \n \r etc have the usual C meaning (bell, backspace, newline, 
carriage return etc)
- Class Shorthands: \d is the same as [[:digit:]]
- positive lookahead
- negative lookahead

The places in KiCad 4.0.1 where they are used are:

./eeschema/class_netlist_object.cpp:static wxRegEx busLabelRe( wxT( 
"^([^[:space:]]+)(\\[[\\d]+\\.+[\\d]+\\])$" ), wxRE_ADVANCED );
./pcbnew/dialogs/dialog_fp_lib_table.cpp:wxRegEx re( wxT( 
".*?\\$\\{(.+?)\\}.*?" ), wxRE_ADVANCED );
./pcbnew/netlist_reader.cpp:wxRegEx reOrcad( wxT( "(?i)[ ]*\\([ \t]+{+" ), 
wxRE_ADVANCED );
./pcbnew/netlist_reader.cpp:wxRegEx reLegacy( wxT( "(?i)#[ \t]+EESchema[ 
\t]+Netlist[ \t]+" ), wxRE_ADVANCED );
./pcbnew/netlist_reader.cpp:wxRegEx reKicad( wxT( "[ ]*\\(export[ ]+" ), 
wxRE_ADVANCED );

As you can see the "advanced" features are actually used in the eeschema 
class_netlist_object (trivially, just the shorthand) and in the netlist reader 
in two places (Escapes and some mysterious "(?i)" moniker - see 
http://www.regular-expressions.info/modifiers.html . Not sure whether that 
would be supported in POSIX extended regular expressions). As far as I can see 
the other places don't actually use the "advanced" features. Note that POSIX 
extended regular expressions are "almost an exact" subset of "advanced regular 
expressions", according to the docs.

Re: Advanced Config Template

[Ludovic Courtès]

Omar Radwan toxemicsqui...@gmail.com skribis:

 I'm trying to install guix in a way that I have X, tor and networking on
 the fly right after install, but I'm having a very hard time with the
 declaration services. This is my config.scm

 (use-modules (gnu))

 (operating-system
   (host-name Atom)
   (timezone US/Pacific)
   (locale en_US.UTF-8)
   (bootloader (grub-configuration (device /dev/sda)))
   (file-systems (cons (file-system
 (device /dev/sda1)
 (mount-point /)
 (type ext4))
   %base-file-systems))
   (users (list (user-account
 (name omar)
 (group users)


 (supplementary-groups '(wheel
 audio video))
 (home-directory /home/omar)

This is good.  Now you need to declare what services in addition to the
base services you’d like to run.

To do that, add a ‘services’ field in the ‘operating-system’ declaration
above (see
http://www.gnu.org/software/guix/manual/guix.html#Using-the-Configuration-System):

  (services (cons* (tor-service)
   (slim-service)
   %base-services))

‘slim-service’ is called this way because it starts Xorg with the SLiM
login manager; see
http://www.gnu.org/software/guix/manual/guix.html#X-Window.

HTH!

Ludo’.

Advanced Config Template

[Omar Radwan]

I'm trying to install guix in a way that I have X, tor and networking on
the fly right after install, but I'm having a very hard time with the
declaration services. This is my config.scm

(use-modules (gnu))

(operating-system
  (host-name Atom)
  (timezone US/Pacific)
  (locale en_US.UTF-8)
  (bootloader (grub-configuration (device /dev/sda)))
  (file-systems (cons (file-system
(device /dev/sda1)
(mount-point /)
(type ext4))
  %base-file-systems))
  (users (list (user-account
(name omar)
(group users)


(supplementary-groups '(wheel
audio video))
(home-directory /home/omar)




I'm not sure how I would exactly be able to do it, but I barely know
anything about Scheme and how its functions work, I hope I'm not
wasting anyones time by asking a stupid question like this, and I hope
someone would be able to help me.