Re: Emacs and URLs in Git commit messages

[Chris Marusich]

Hi,

Thank you for the replies!

Maxime Devos  writes:

> I don't known any emacs command for that, but you inspired me to write
> such a command myself: [1].
>
> Maxime.
> [1]: 
> https://notabug.org/mdevos/things/commit/b0400ba06b6f031e88f1f89b47079c3c6d7dcac4

zimoun  writes:

> I am not representative since I commit few fixes.  Well, I have a tiny
> helper that pushes to the kill ring:
>
> 
>
> Especially reading bug#45314 in Debbugs mode, I type “M-x
> my/guix-issues” then the URL ’http://issues.guix.gnu.org/issue/45314’ is
> stashed.  If not in debbugs mode, as here in message mode, the tiny
> helper asks the bug number then pushes it to the kill ring.  This tiny
> helper is far from perfect, any improvements is welcome. :-) Especially
> if it is already provided by an Emacs command.
>
> About the brackets, I type them.

Ludovic Courtès  writes:

> I have this helper for debbugs.el:
>
> (defun ludo-copy-debbugs-url ()
>   "Add to the kill ring the URL of the Debbugs issue at point."
>   (interactive)
>   (let ((url1 (concat "https://bugs.gnu.org/;
> (number-to-string (debbugs-gnu-current-id
>   (url2 (concat "https://issues.guix.gnu.org/;
> (number-to-string (debbugs-gnu-current-id)
> (kill-new url1)
> (kill-new url2)
> (message "Copied %s and %s" url1 url2)))
>
> (define-key debbugs-gnu-mode-map (kbd "C-w") 'ludo-copy-debbugs-url)
>
> That way I can C-w on a bug in *Guix Bugs* and I get the two URLs in the
> clipboard (I normally use “bugs.gnu.org” as the canonical bug URL.)
>
> Ludo’.

OK, I see.  So I'm not missing out on some built-in Emacs (or
debbugs-emacs) magic; most people just put together something convenient
on their own.  That makes sense!  Thank you for the examples; it's
helpful to see how others are doing it.  I think I'll try something
similar.

Bengt Richter  writes:

> I am not sure I understand your context or goal in searching ;/

I'm always interested in finding more effective ways to get things done.
Sometimes, asking people how they do something is the best way to
discover new methods, even if the answer seems simple or obvious.

-- 
Chris


signature.asc
Description: PGP signature

Re: Unreproducible “guix pack -f docker” because config.scm-builder

[zimoun]

Hi Ludo,

On Fri, 05 Feb 2021 at 11:09, Ludovic Courtès  wrote:

> So I guess you can propose a patch and let someone else review it.
> :-)

I will. :-)


> Looks like tar made this file a hard link in one case and not in the
> other.  This is weird because we don’t ask it to create hard links
> (there’s even a comment in (guix scripts pack)).

I have 2 machines running Debian and one running Ubuntu.  The 2 Debian
produces the same things.  Ubuntu not.


> Is this docker image the result of the same derivation?  Could you try
> building that derivation on different machines?  (You can copy the .drv
> around with ‘guix copy’.)

It was built with different derivations.  I mean the scenario where
Alice wants to rebuild what Bob did.

Now, let use the same derivation.  Some details in case I am doing
something wrong:

--8<---cut here---start->8---
$ echo Ubuntu

$ guix gc -D $(guix pack -f docker hello)
finding garbage collector roots...
[0 MiB] deleting
'/gnu/store/nkvlqwzvxdlhzlc7vhfcngxc19x2ay2f-docker-pack.tar.gz'
deleting `/gnu/store/trash'
deleting unused links...
  C-c C-c
 
$ guix gc -D $(guix pack -f docker hello -d)
finding garbage collector roots...
[0 MiB] deleting
'/gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv'
deleting `/gnu/store/trash'
deleting unused links...
  C-c C-c
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ echo Debian

$ guix copy $(guix pack -f docker hello -d) --to=meary
guix copy: sending 1 store item (0 MiB) to '193.48.40.110'...
/gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ echo Ubuntu

$ guix build /gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv
substitute: mise à jour des substituts depuis « https://ci.guix.gnu.org»... 
100.0 %
La dérivation suivante sera compilée :
/gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv  
 
construction de
/gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv...
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
construction de
/gnu/store/323k33sfx869d0nkh69ary8sj6xiy4s4-docker-pack.tar.gz.drv réussie
/gnu/store/nkvlqwzvxdlhzlc7vhfcngxc19x2ay2f-docker-pack.tar.gz  
--8<---cut here---end--->8---

So tar is appearing here… Hum?!  However, if I redo the same steps, it
does not.  Well, I do not like that…

Building on Ubuntu using the derivation from Debian gives the same image
as building on Ubuntu using the derivation from Ubuntu.

I thought the tools were captured by the commit: same commit, same
tools.  Well, I miss something…

> Could you also show the output of:
>
>   stat 
> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
>
> on the two machines you used?

First on Debian and second on Ubuntu

--8<---cut here---start->8---
$ stat 
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
  File: 
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
  Size: 29960   Blocks: 64 IO Block: 4096   regular file
Device: 801h/2049d  Inode: 8129616 Links: 5
Access: (0555/-r-xr-xr-x)  Uid: (0/root)   Gid: (0/root)
Access: 2021-02-05 17:37:15.728728952 +0100
Modify: 1970-01-01 01:00:01.0 +0100
Change: 2020-06-17 12:40:06.389935679 +0200
Birth: -
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ stat 
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
Fichier : 
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
Taille : 29960   Blocs : 64 Blocs d'E/S : 4096   fichier
Périphérique : 814h/2068d   Inœud : 148900093   Liens : 1
Accès : (0555/-r-xr-xr-x)  UID : (0/root)   GID : (0/ root)
Accès : 2021-02-05 17:46:05.537903382 +0100
Modif. : 1970-01-01 01:00:01.0 +0100
Changt : 2020-12-04 23:16:33.155711694 +0100
Créé : -
--8<---cut here---end--->8---
  


> I wonder if it could be that tar nowadays decides to preserve hard links
> by default and one of your machine had this file hard-linked while the
> other one didn’t.

Wow, I thought that the same tar was used the one provided by Guix and
not by the host.  For sure the default tar are not the same on both
machine (v1.30 for Debian and v1.29 for Ubuntu).

What is the md5sum checksum of ’guix pack -f docker hello’ on Guix
System for commit b9a54aa?

Thanks for the help,
simon

Re: Blog post about the upcoming FOSDEM + Guix Day

[Leo Famulari]

On Fri, Feb 05, 2021 at 11:37:49AM +0100, Ludovic Courtès wrote:
> On IRC Leo (American timezone) was thinking about having a session to
> discuss branching strategies, and there were also discussions about
> substitute availability and continuous builds for QA.

Yes, I think we can discuss branching and CI / QA in the same session.

I'm in the New York time zone, UTC−05:00

By the way, can we record these sessions?

Re: Potential security weakness in Guix services

[Maxime Devos]

On Fri, 2021-02-05 at 13:20 +0100, Maxime Devos wrote:
> On Fri, 2021-02-05 at 10:57 +0100, Ludovic Courtès wrote:
> > [...]
> [...]
> 
> I'll try to implement this API in Scheme (using the FFI), and post
> it at https://notabug.org/mdevos/things.  I'll post a follow-up
> messsage once I've implemented the basics (openat, chmodat,
> chownat).

Ping!
https://notabug.org/mdevos/things/src/a0715e6758ad43252e16993dcf688a25156057f3/fs-at.scm



signature.asc
Description: This is a digitally signed message part

Re: Potential security weakness in Guix services

[Maxime Devos]

On Fri, 2021-02-05 at 10:57 +0100, Ludovic Courtès wrote:
> Hi Maxime,

> 
> > I don't know how I should implement this properly in Guile, though.
> > In C, I would use loop using openat with O_NOFOLLOW, in combination
> > with stat, but Guile doesn't have openat or O_NOFOLLOW.
> 
> In this case we need a solution without openat for now.  Perhaps simply
> changing ‘mkdir-p/perms’ to ‘lstat’ components as it goes?

A compromised service could create a component as a regular file or
directory, and quickly replace it with a symlink after the activation
gexp checks the component wasn't a symlink but before the chown or
chmod.

It's a tiny window though.  I was going to say this tiny ‘exploitation
window’ could be ignored for now until an API to openat & other *at
C functions makes it into guile, but then I checked the inotify(7)
API.  Apparently, inotify events are generated for directory accesses.

> > [...]
> > I'll look into writing a concrete proposal for *at in guile.
> > I'll post a link to the guile mailing list message when it has
> > been composed and sent.

Link: https://lists.gnu.org/archive/html/bug-guile/2021-02/msg2.html

> The difficulty in designing such an interface is that the Scheme API is
> more about ports than it’s about file names and file descriptors.

This doesn't seem a large issue to me.
Ports for directories can be made with 'open': (open "/home" O_RDONLY).
Example use of for changing permission bits of "/home/USER" with
proposed API:

(let* ((directory (open "/home" O_RDONLY))
   ;; in the C API, this translates to:
   ;; openat(directory-fd, "USER", O_IDK)
   ;; (replace O_IDK with appropriate open flags)
   (file-in-dir (open  (make-path-at directory "USER") O_IDK)))
  (chmod file-in-dir #o077)
  (chown file-in-dir 0 0))

I'll try to implement this API in Scheme (using the FFI), and post
it at https://notabug.org/mdevos/things.  I'll post a follow-up
messsage once I've implemented the basics (openat, chmodat,
chownat).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Emacs and URLs in Git commit messages

[Ludovic Courtès]

Hi,

Chris Marusich  skribis:

> Many Guix commits look like this:
>
>   commit f9978346e73359ac1d8b88c9ed874edc7225582b
>   Author: Ludovic Courtès 
>   Date:   Fri Dec 18 18:10:04 2020 +0100
>
>   avahi: Remove poll timeout when possible.
>
>   Fixes .
>
>   * guix/avahi.scm (avahi-browse-service-thread): Change timeout default 
> value
>   to false when no "stop-loop?" procedure is passed. Adapt 
> "iterate-simple-poll"
>   call accordingly.
>
>   Signed-off-by: Mathieu Othacehe 
>
> Regarding the URL, do people just type it all out, including the opening
> and closing brackets (<>)?  Or is there an Emacs command that does it
> for you?  I've briefly looked on the Internet, but this is the kind of
> thing that seems difficult to search for.

I have this helper for debbugs.el:

--8<---cut here---start->8---
(defun ludo-copy-debbugs-url ()
  "Add to the kill ring the URL of the Debbugs issue at point."
  (interactive)
  (let ((url1 (concat "https://bugs.gnu.org/;
  (number-to-string (debbugs-gnu-current-id
(url2 (concat "https://issues.guix.gnu.org/;
  (number-to-string (debbugs-gnu-current-id)
(kill-new url1)
(kill-new url2)
(message "Copied %s and %s" url1 url2)))

(define-key debbugs-gnu-mode-map (kbd "C-w") 'ludo-copy-debbugs-url)
--8<---cut here---end--->8---

That way I can C-w on a bug in *Guix Bugs* and I get the two URLs in the
clipboard (I normally use “bugs.gnu.org” as the canonical bug URL.)

Ludo’.

Re: Questions regarding Python packaging

[Hartmut Goebel]

Am 23.01.21 um 13:34 schrieb Lars-Dominik Braun:

Remove pip and
setuptools from python (saves almost 20MiB from the closure


When doing to we need to be carefully. pip is expected to be available 
after installing "python". So when removing pip and setuptool, we would 
need some "python/bootstrap" package without pip and setuptools and some 
"python" package still including both.



--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Blog post about the upcoming FOSDEM + Guix Day

[Ludovic Courtès]

Hi,

Pjotr Prins  skribis:

> We start Monday at 5am UTC (6am AMS/Berlin, 7am Athens) to cater for
> some of Asia. The day ends when the last one switches off the lights
> :).  Manolis, Efraim and Bonface have promised to be there. The
> bluebutton link will be announced a day ahead.

Could you update ?

Are there planned sessions and associated facilitators?  I think the
things on the wiki page are copy/pasted from last year and should be
updated.

I’m happy to have be on a “state of the union” session with the other
co-maintainers but we’ll need to know in advance (like now :-)) so we
can structure our thoughts a bit.  (Timezones would be Europe +
America.)

On IRC Leo (American timezone) was thinking about having a session to
discuss branching strategies, and there were also discussions about
substitute availability and continuous builds for QA.

(But really, I don’t think I can make it at 6AM local time…)

Thanks for organizing this!

Ludo’.

Re: 03/163: build/python: Add a new guix-pythonpath procedure.

[Hartmut Goebel]

Hi Maxim,

many thanks for picking up this issue.

Indeed, I thought about the possibility to filter the GUIX_PYTHONPATH
entries based on their version at runtime after I wrote my initial
reply.  It makes life easier.  I've updated the
cu/farewell-to-pythonpath branch with this new way of doing things.


I had a look at the first changes (only) and have some remarks:

1) Did I understand this correctly: `sitecustomize.py` is filtering 
GUIX_PYTHONPATH for all parts containing "/lib/pythonX.Y/site-packages" 
(with X.Y being Major.Minor)?


2) This does not remove duplicates and does not honor .pth files in the 
respective directories - which might still be used. Thus 
site.addsitedir() should be called for adding the paths. This also takes 
care about duplicates.


3) Empty part (…::…) are not handled.

4) Since PYTHONPATH is evaluated prior to importing sitecustomize, any 
sitecustominze.py in the user's path will overwrite our file, thus 
inhibiting our paths to be added. Not sure this is what we want in Guix.


5) When implementing the logic into site.py, the code could be 
simplified, Eg. You could patch a "getguixsitepackages" (based on 
getsitepackages) and a "addguixsitepackages" (based on 
addguixsitepackages) into site.py. Downside is that maybe we need 
different patches for different Python versions. Benefit is simpler code 
- no need to search the correct position in the list.


6) Please add some more comments to the code explaining the idea.


Some nitpicking:

> python_root = os.path.realpath(sys.executable).split('/bin/')[0]

There already is sys.prefix, which is also the base for the entry in 
sys.path (see top of site.py


> major_minor = '{}.{}'.format(sys.version_info[0], sys.version_info[1])

major_minor = '{}.{}'.format(*sys.version_info)

>sys.path = sys.path[:index] + matching_sites + sys.path[index:]

sys.path[index:index] = matching_sites


I suggest using os.path.join(), os.path.split(), os.pathsep, etc. to be 
forward-compatible. Imagine we want to port Guix to another platform 
with different separators.


--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Unreproducible “guix pack -f docker” because config.scm-builder

[Ludovic Courtès]

Hi,

zimoun  skribis:

> then the sysconfdir is set to /usr/local/etc because it is the default.
> And so it leads to subtle differences really hard to guess.  I think it
> is worth to add one sentence or footnote at the end of the section
> «Running Guix Before It Is Installed», right after:
>
> Note that ./pre-inst-env guix pull does not upgrade the local
> source tree; it simply updates the ~/.config/guix/current
> symlink (see Invoking guix pull). Run git pull instead if you
> want to upgrade your local source tree.
>
> Something like: «Note that ’guix pull’ preserves the settings of the host
> Guix, for instance ’sysconfdir’, and by default the GNU standards set
> ’prefix’ to ’/usr/local/’ and ’sysconfdir’ to ’$prefix/etc’, whereas
> regular Guix uses ’--sysconfdir=/etc/’.»
>
> WDYT?

As often, I have mixed feelings: we would end up +/- duplicating the
Standards in the manual, possibly even without citing the primary source
(I could be a Wikipedian :-)).  So to me that’s not great.

OTOH, as you write, letting people stumble upon this kind of issue is
not an option, either.

So I guess you can propose a patch and let someone else review it.  :-)

>> You did find other differences eventually though, right?
>
> The produced tarballs have the same Guix hash, i.e., all the same
> inputs, but not the same outputs, compare with commit b9a54aa:

[...]

> --- 
> /tmp/docker-meary/4ca83868d5e98cb06179a2a7372afe029c10d43bdc9fbfcc5771b89da74889b8/layer.tar
> +++ 
> /tmp/docker-pfiuh02/4ca83868d5e98cb06179a2a7372afe029c10d43bdc9fbfcc5771b89da74889b8/layer.tar
> ├── file list
> │ @@ -823,17 +823,17 @@
>
> [...]
>
> │ --r-x… 29960 
> gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64
>
> [...]
>
> │ +hr-x… 0 
> gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64

Looks like tar made this file a hard link in one case and not in the
other.  This is weird because we don’t ask it to create hard links
(there’s even a comment in (guix scripts pack)).

Is this docker image the result of the same derivation?  Could you try
building that derivation on different machines?  (You can copy the .drv
around with ‘guix copy’.)

Could you also show the output of:

  stat 
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/libexec/getconf/POSIX_V6_LP64_OFF64

on the two machines you used?

I wonder if it could be that tar nowadays decides to preserve hard links
by default and one of your machine had this file hard-linked while the
other one didn’t.

Ludo’.

Re: Potential security weakness in Guix services

[Ludovic Courtès]

Hi Maxime,

Maxime Devos  skribis:

> On Tue, 2021-02-02 at 14:07 +0100, Ludovic Courtès wrote:
>> OK, I see.  Roughly, this symlink chown story would be a local exploit
>> that the attacker can take advantage of after exploiting the RCE to
>> potentially get root access.
>> 
>> ‘mkdir-p/perms’ could check that the directory is not a symlink, to
>> begin with.  Is this what you had in mind, Maxime?
>
> Yes!  Though the parent- and grandparent and etc. should be checked as well.
> If e.g. (I don't have a real example at hand) knot's activation has
> a (mkdir-p/perms "/var/lib/knot/e/t/c/e/t/e/r/a" uid/gid-stuff) line, then
> mkdir-p/perms has to take care that the "e", "t", "c", "e", "t", "e, "r"
> and "a" directories aren't symlinks.

OK.

> I don't know how I should implement this properly in Guile, though.
> In C, I would use loop using openat with O_NOFOLLOW, in combination
> with stat, but Guile doesn't have openat or O_NOFOLLOW.

In this case we need a solution without openat for now.  Perhaps simply
changing ‘mkdir-p/perms’ to ‘lstat’ components as it goes?

> I've proposed adding the O_NOFOLLOW to guile [1].  I don't have a
> proposal for openat in guile yet.  If I do propose an interface
> to openat(2); I should probably make a proposal for fchownat
> and other *at variants at the same time.  I don't have a concrete
> proposal for how a nice Scheme interface would look like.
>
> In the mean time, I suppose it should be possible to use openat
> via the FFI and define O_NOFOLLOW manually in Guix.
>
> I'll look into writing a concrete proposal for *at in guile.
> I'll post a link to the guile mailing list message when it has
> been composed and sent.

I think it’s a good endeavor, but it’s a longer-term one since it’ll
take some time before this new version is in use by all the Guix code.

The difficulty in designing such an interface is that the Scheme API is
more about ports than it’s about file names and file descriptors.

Thanks!

Ludo’.