Re: Adding the maven-build-system

[Leo Famulari]

On Sat, Apr 04, 2020 at 05:52:37PM +0200, Julien Lepiller wrote:
> Before I send a patch series (~200 patches if i count well),

It will be more convenient for reviewers if you pushed a
wip-maven-build-system branch to Savannah.

Re: Adding the maven-build-system

[Jack Hill]

On Sat, 4 Apr 2020, Julien Lepiller wrote:


Hi Guix!

great news! I have successfully built a hello world package with a
fully bootstrapped maven-build-system!  This email will summarize a bit
what I did and how I got there.  For the impatient, my work is
currently at https://framagit.org/tyreunom/maven-build-channel. You can
build maven-test which only depends on the maven build system, which in
turn depends only on packages in maven-boostrap.scm and
maven-plugins-boostrap.scm.

Before I send a patch series (~200 patches if i count well), I would
like to describe what I want to do and get a bit of feedback.  I feel
that this work is good enough to go to guix, but it might bring it to
an unconfortable state where many packages will need to be converted to
the new build system(s).  I'd like to hear your opinion!


Julien,

I don't have any useful comments at this time, but I wanted to 
congratulate you on this progress. I know that it must have been a lot of 
work. Thank you for the code and the excellent description. I look 
forward to trying the build system soon.


Best,
Jack

Adding the maven-build-system

[Julien Lepiller]

Hi Guix!

great news! I have successfully built a hello world package with a
fully bootstrapped maven-build-system!  This email will summarize a bit
what I did and how I got there.  For the impatient, my work is
currently at https://framagit.org/tyreunom/maven-build-channel. You can
build maven-test which only depends on the maven build system, which in
turn depends only on packages in maven-boostrap.scm and
maven-plugins-boostrap.scm.

Before I send a patch series (~200 patches if i count well), I would
like to describe what I want to do and get a bit of feedback.  I feel
that this work is good enough to go to guix, but it might bring it to
an unconfortable state where many packages will need to be converted to
the new build system(s).  I'd like to hear your opinion!



First of all, a bit of background on maven.  It wants to download
dependencies described in the pom.xml file in a local repository.  The
maven-build-system passes the -o (--offline) flag to ensure nothing is
downloaded.  Instead, the build fails immediately if maven determines
that it will need to download something that is not locally present.
Passing the -Dmaven.home flag also ensures we also control the location
of the maven repository when building.

Each maven package has a groupId, an artifactId and a version, and the
repository structure reflects that: from the maven.home directory, a
jar artifact will be located at:
.m2/repository/the/group/id/the-artifact-id/version/the-artifact-id-version.jar

Additionally, maven relies on the presence of a .pom file in that
repository, in the same location:
.m2/repository/the/group/id/the-artifact-id/version/the-artifact-id-version.pom

When building, maven will look at the pom file to get a list of
dependencies (listed in dependencies, and in plugins).  Without the pom
file, some dependencies might be missing while building, running the
tests, etc. However, dependencies in pom files also come with a
version, and maven will refuse to load any other version, even if the
specified version is not present, but a more recent version is present.

The first patch of the series I propose will introduce
https://framagit.org/tyreunom/maven-build-channel/-/blob/master/maven/build/maven/pom.scm,
which contains a parser for the pom.xml as well as a procedure to
override dependency and (optionally) plugin version numbers by versions
present in the guix build environment. This override depends on the
presence of the dependencies in the environment, at a well-known
location (a one-to-one mapping with a maven repository).  This patch
will also introduce two new procedures in
https://framagit.org/tyreunom/maven-build-channel/-/blob/master/maven/build/java-utils.scm:
install-from-pom which applies the transformation to the passed pom,
and installs a jar file (either the one specified in #:jar-name, or the
only jar file present in the build directory) along with the pom file
in a directory structure that reflects maven repository structures:
lib/m2/the/group/id/the-artifact-id/version/the-artifact-id-version.{jar,pom}.

Subsequent patches will move existing package definitions from java.scm
to maven-bootstrap.scm (including junit and some other packages with
more than that purpose), until we have moved maven itself.  During that
move, some packages will be updated (for instance junit) and the junit
bootstrap is completed (I remove the dependency on maven and osgi
completely because we don't build the osgi bundle and maven plugin
properly anyway).  Additional packages will be included (see packages
whose name end in "-pom") to add so-called "parent-poms" referenced in
packages' pom.xml.  I'm not entirely sure if that is required (we could
maybe remove the parent information, but it doesn't seem to be always
the case upstream).  The install-pom-file procedure in java-utils.scm
is tasked with that job: it will install the pom file in the expected
directory structure, in lib/m2.

Since maven itself cannot build anything, it requires some plugins to
do the work (maven is an artifact resolver, but it's not a build tool
by itself).  In order to build the hello-world package, I need at least
these plugins: the maven-resources-plugin (to collect
src/main/java/resources), the maven-compiler-plugin (to actuall build
the packages, it creates .class files in target/build), the
maven-jar-plugin (to create the jar file), the maven-surefire-plugin
(to run the tests using junit) and the maven-install-plugin (to install
the plugin in the final location).  Additionally, the
maven-enforcer-plugin seems to be used by many, so I built it, but it's
probably not required; I'll see if I can omit it for now.

Subsequent patches will add packages in maven-plugins-bootstrap.scm.
These are mostly new packages that will still use the ant-build-system.
They are dependencies of the plugins.  Plugins themeselves are built
there and require a special phase (generate-plugin.xml) to generate yet
another XML file.  In order to recognize its plugins, 

Re: Proxy settings wrt guix daemon

[Mathieu Othacehe]

Hello Vicent,

>
> What is the UI to run that guix-daemon --set-proxy ?
> Use herd set-proxy guix-daemon "https://proxy:3128; ?
> (Like what is done for mcron)
>
> I really need a few "make this thing here do that" hints...
>
> And then also a bit of a hint on how I would test the
> modifications without risking my running system.

>From what I understand, the guix-daemon process is able to propagate
some of its environments variable to the builder. It seems that
http_proxy variable (and friends), are set as impureEnvVars by Guix. You
can search for '#:leaked-env-vars' over the Guix sources.

So it seems to indicate that if the guix-daemon process has the
http_proxy environment variable set, it will handle it correctly. So the
question is how to be able to set this variable dynamically?

I don't think there's a way to change the environment variables of a
process from the outside. So unless you come up with a new mechanism
inside guix-daemon, able to read proxy parameters from a file or so (not
desirable), the variable needs to be set before guix-daemon is launched.

Implementing what's proposed by Ludo would then make the following
command: 'herd set-proxy guix-daemon "https://proxy:3128;' do the
following things:

--8<---cut here---start->8---
(setenv "HTTP_PROXY" "https://proxy:3128;)
(restart guix-daemon) ; not the appropriated syntax
--8<---cut here---end--->8---

I don't know if that's what Ludo has in mind but I can't see any other
ways to do it.

Now Ludo has also proposed another solution here[1]. This solution
would require that (guix script perform-download) and (guix script
substitute) honor what would look like a 'build-option' (see
set-build-options in (guix store)). But this seems harder to implement.

Mathieu

[1]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25569

Re: Unencrypted boot with encrypted root

[pelzflorian (Florian Pelz)]

Thank you Ellen for the good summary.  I knew most but in less detail.

On Sat, Apr 04, 2020 at 10:12:46AM +0200, Ellen Papsch wrote:
> Am Freitag, den 03.04.2020, 21:44 +0200 schrieb pelzflorian (Florian
> Pelz):
> > But it would still be possible for an attacker […]
> Yes, though it should not be so easy like with unprotected /boot
> partition.

So the better solution would be a better GRUB instead of unprotected
/boot.  Could key files help in passing the passphrase on to the Linux
kernel?  The Arch Wiki says this:
.

Keyboard layouts then will only all work properly everywhere once
fixed in GRUB.

Regards,
Florian

Re: [BLOG] On migration to the Hurd

[Jan Nieuwenhuizen]

Rene writes:

> Hello Ludovic, Jan,
>
> About the bootstrap ext2fs.static server, it seems that it is also
> required to create a link in /libexec, attached a patch.

Ah...great!

> and now it shows another error:
> --
>  Hurd server bootstrap: ext2fs[device:hd0s1] exec startup proc auth.
>  /libexec/console-run: /dev/console: Permission denied
>  /libexec/console-run: cannot execute /libexec/runsystem: Exec format 
> error
> --
>
> I'll keep checking ..

With the attached patches (I added a copyright line for you), I now get
a prompt!

Greetings,
janneke

PS: you can also pull wip-hurd-vm from https://gitlab.com/janneke/guix.git

>From 56f67343c4108da8c8f0a91844aebd28181dc912 Mon Sep 17 00:00:00 2001
From: Rene Saavedra 
Date: Fri, 3 Apr 2020 19:31:01 -0600
Subject: [PATCH 1/3] system: hurd: Add symlink for `../hurd/libexec.`

* /gnu/system/hurd.scm (hurd-directives): Add symlink in
`/libexec/console-run` for `../hurd/libexec`.
---
 gnu/system/hurd.scm | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gnu/system/hurd.scm b/gnu/system/hurd.scm
index 586bfa0fca..97c54a2992 100644
--- a/gnu/system/hurd.scm
+++ b/gnu/system/hurd.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Ludovic Courtès 
+;;; Copyright © 2020 Rene Saavedra 
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -94,7 +95,11 @@ menuentry \"GNU\" {
   ("/hurd" -> ,(file-append (with-parameters ((%current-target-system
"i586-pc-gnu"))
   hurd)
-"/hurd"
+"/hurd"))
+  ("/libexec" -> ,(file-append (with-parameters ((%current-target-system
+  "i586-pc-gnu"))
+ hurd)
+   "/libexec"
 
   (qemu-image #:file-system-type "ext2"
   #:file-system-options '("-o" "hurd")
-- 
2.26.0

>From 248e44e7f42312c92b23d2bc3d10a80a5bb2eadf Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" 
Date: Sat, 4 Apr 2020 10:32:22 +0200
Subject: [PATCH 2/3] gnu: hurd: Have scripts use store file names.

* gnu/packages/hurd.scm (hurd)[inputs]: Add bash-minimal, coreutils, sed.
[arguments]: Use them in new `use-store-file-names' stage.
---
 gnu/packages/hurd.scm | 38 --
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/hurd.scm b/gnu/packages/hurd.scm
index 8f087172de..09b09b6397 100644
--- a/gnu/packages/hurd.scm
+++ b/gnu/packages/hurd.scm
@@ -30,6 +30,7 @@
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
   #:use-module (gnu packages autotools)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages flex)
   #:use-module (gnu packages bison)
   #:use-module (gnu packages perl)
@@ -361,6 +362,36 @@ boot, since this cannot be done from GNU/Linux."
(copy-file rc file)
(substitute* file
  (("@HURD@") out))
+   #t)))
+ (add-after 'install 'use-store-file-names
+   (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((out  (assoc-ref outputs "out"))
+(bash (assoc-ref inputs "bash"))
+(coreutils (assoc-ref inputs "coreutils"))
+(sed (assoc-ref inputs "sed")))
+   (substitute* (list (string-append out "/libexec/runsystem")
+  (string-append out "/libexec/runsystem.hurd") ; Hmm?
+  (string-append out "/sbin/MAKEDEV")
+  (string-append out "/bin/fakeroot")
+  (string-append out "/bin/remap")
+  (string-append out "/bin/sush"))
+ ;; huh, why are shebangs using the build system's bash?
+ (("^#!.*/bash") (string-append "#! " bash "/bin/bash"))
+ (("^#!.*/sh") (string-append "#! " bash "/bin/bash"))
+ (("=/libexec") (string-append "=" out "/libexec"))
+ (("^SHELL=/bin/sh")
+  (string-append "SHELL=" bash "/bin/bash"))
+ (("^PATH=")
+  (string-append "PATH=" out "/bin"
+ ":" out "/sbin"
+ ":" coreutils "/bin"
+ ":" sed "/bin"
+ ":/run/current-system/profile/bin"
+ ":/run/current-system/profile/sbin"))
+ )
+   (substitute* (string-append out "/bin/sush")
+ (("-/bin/fmt") (string-append "-/" coreutils "/bin/fmt"))
+ (("-/bin/login") (string-append "-/" coreutils "/bin/legin")))
#t
#:configure-flags (list 

Re: Unencrypted boot with encrypted root

[Ellen Papsch]

Am Freitag, den 03.04.2020, 21:44 +0200 schrieb pelzflorian (Florian
Pelz):
> 
> So using a single encrypted partition instead of separate /boot
> protects from script kiddies (siblings/“friends”?) with hardware
> access that know how to put their own grub.cfg on an unencrypted
> /boot
> partition and then wait for you to unsuspectingly use your machine.
> 

Yes, it is better known as "evil maid attack", because the original
thought included a hotel room[0].

> But it would still be possible for an attacker to flash or replace
> the
> motherboard’s UEFI, or perhaps the part of GRUB installed on the
> unaltered motherboard would willingly load a manipulated hard disk?
> Or just install a keylogger.
> 

Yes, though it should not be so easy like with unprotected /boot
partition.

You have these boot stages in a modern UEFI system (just numbered
sequentially):

- hardware initialization
- stage 0, which is a minimal bootloader including the Secure Boot key
on ROM.
- stage 1, which is Management Engine on Intel platforms or Platform
Security Processor on AMD platforms. Some of it is on ROM, while most
can be (not easily) flashed. ME is a Minix derivate with its own little
processor (ARM IIRC).
- stage 2, which is your UEFI BIOS.
- stage 3, which is the program that gets put in the /boot/efi
directory and registered with the BIOS, i.e. GRUB.
- stage 4-n, Guix!

If you are interested in the flaws of stage 1, check out [1] and [2].
[3] is very interesting too, as it not only presents hardware flaws but
also suggests possible way forward. (These are the same video URLs I
initially posted.)

Breaking any earlier stage gives you control over the later stages.

The general gist is that all common (consumer) hardware is flawed and
with it the software that runs on it. That makes free hardware ever
more important. It's also why people are interested in breaking stage
1; not so much for attack, but because it is closely linked to the
hardware and prevents their freedom.

Regards,
Ellen

[0] https://en.wikipedia.org/wiki/Evil_maid_attack
[1] https://media.ccc.de/v/36c3-10694-intel_management_engine_deep_dive
[2] 
https://media.ccc.de/v/thms-38-dissecting-the-amd-platform-security-processor
[3] 
https://media.ccc.de/v/36c3-10690-open_source_is_insufficient_to_solve_trust_problems_in_hardware

Re: Adding a %desktop-packages

[Brice Waegeneire]

Hello John,

On 2020-04-04 00:22, John Soo wrote:

I raised the idea of providing smaller lists of packages that might go
well together instead of one large %desktop-packages. One reason to do
this, for instance, might be to not make someone who wants to use btrfs
always import the ext4 packages. Or not lock someone into using 
nettools

if they are using iproute2, etc.


Regarding the file-system utils packages I'm working on a patch adding 
them
to the system profile based on the type of file-system defined. So, in 
the
end, `e2fsprogs`, `btrfs-progs` and such will be removed from 
%base-packages.


Also I've just submitted a patch[0] to split %base-packages as we talked
about on IRC a couple of weeks ago.

[0]: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=40423

- Brice

Re: [BLOG] On migration to the Hurd

[Rene]

Hi,

>
> and now it shows another error:
> --
>  /libexec/console-run: /dev/console: Permission denied

for this error it is required to create the devices with:

./MAKEDEV console tty ...

I did this manually in Debian and the error no longer shows.


Rene

New Russian PO file for 'guix-manual' (version 1.1.0-pre1)

[Translation Project Robot]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'guix-manual' has been submitted
by the Russian team of translators.  The file is available at:

https://translationproject.org/latest/guix-manual/ru.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

https://translationproject.org/latest/guix-manual/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

https://translationproject.org/domain/guix-manual.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

The Translation Project robot, in the
name of your translation coordinator.

Re: Adding a %desktop-packages

[Gábor Boskovits]

Hello,

John Soo  ezt írta (időpont: 2020. ápr. 4., Szo 2:23):

> Hi there,
>
> I am on board with providing some predefined lists of packages.
>
> I raised the idea of providing smaller lists of packages that might go
> well together instead of one large %desktop-packages. One reason to do
> this, for instance, might be to not make someone who wants to use btrfs
> always import the ext4 packages. Or not lock someone into using nettools
> if they are using iproute2, etc.
>
> Similarly, I think that many users, myself included, use a manifest file
> to manage user packages. It would help to have finer grained
> package lists so that the manifests could reuse them and not be
> requiring system basics along with it.
>
> What do you think?
>

This is more in line with my thoughts. Also, if we have some of these fine
grained lists, it would be easy to provide collections of these, so we can
do both things, but in a more useful way.

>
> - John
>

Best regards,
g_bor

>