Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi Ludo,

>> Making this work would need two things:
>> 1) not stripping unexisting directories from search paths
>> 2) constructing environment variables prepending every profile root to
>>the search paths of each profile.
>>
>> Now, my question is, why are unexisting search paths removed?
>
> I don’t think this is really the problem here.  The problem here is that
> if a profile contains only, say, guile-json, there is no way Guix can
> know about GUILE_LOAD_PATH because that variable is associated with
> Guile itself, not with guile-json.

The idea is each profile environment unconditionally adds search paths
for all the profiles in use (which are passed when evaluating
/etc/profile in the GUIX_PROFILES variable). This was the
proposal in my first message (looks *similar* to an earlier post in the
bug discussion, but it is quite different).

In the case you suggest, the profile cointaining guile will define
GUILE_LOAD_PATH to point inside its own profile and *also* inside the
profile containing guile-json.

The profile that has guile-json will do nothing, as its installer
packages have no search paths.

This is the key:

export ACLOCAL_PATH="${profiles//:/\/share/aclocal:}/share/aclocal

So ACLOCAL_PATH will have an entry for /share/aclocal,
regardless of that path existing or not. If path is not there, no harm
done. If it is (i.e. that profile has a package installing an aclocal
.m4 file) then aclocal will find it.

I will try to prepare a patch, so we can see (and might be there is some
problem I still fail to see).

BR

Carlos

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hi Carlos,

csanchez...@gmail.com (Carlos Sánchez de La Lama) skribis:

> Hi Ludo, Alex,
>
>> In the example Alex gave, none of the two profiles specifies
>> GUILE_LOAD_PATH if taken individually.  This example is also in the
>> manual to illustrate the use of multiple -p flags with --search-paths
>> (info "(guix) Invoking guix package").
>
> I think I understood now. I knew "guix --search-paths" only returned
> those search paths which *do exist* in the current profile (thus the
> need for multiple -p parameters), but I (wrongly) thought search paths
> were added unconditionally to /etc/profile.
>
> Making this work would need two things:
> 1) not stripping unexisting directories from search paths
> 2) constructing environment variables prepending every profile root to
>the search paths of each profile.
>
> And my suggestion only solved point 2.
>
> Now, my question is, why are unexisting search paths removed?

I don’t think this is really the problem here.  The problem here is that
if a profile contains only, say, guile-json, there is no way Guix can
know about GUILE_LOAD_PATH because that variable is associated with
Guile itself, not with guile-json.

Does that make sense?

Thanks,
Ludo’.

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi Ludo, Alex,

> In the example Alex gave, none of the two profiles specifies
> GUILE_LOAD_PATH if taken individually.  This example is also in the
> manual to illustrate the use of multiple -p flags with --search-paths
> (info "(guix) Invoking guix package").

I think I understood now. I knew "guix --search-paths" only returned
those search paths which *do exist* in the current profile (thus the
need for multiple -p parameters), but I (wrongly) thought search paths
were added unconditionally to /etc/profile.

Making this work would need two things:
1) not stripping unexisting directories from search paths
2) constructing environment variables prepending every profile root to
   the search paths of each profile.

And my suggestion only solved point 2.

Now, my question is, why are unexisting search paths removed? This is
done in guix-search-paths.scm(evaluate-sarch-paths), by calling
search-path-as-list, which filters out unexisting directories. Why not
just add all of them? Profiles are immutable once created, so having a
search path pointing into a subexisting subdirectory should not do any
harm.

Am I missing something?

Carlos

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hi,

csanchez...@gmail.com (Carlos Sánchez de La Lama) skribis:

> Alex Kost  writes:

[...]

>>> what about something like:
>>>
>>> - /etc/profile
>>>   # [...]
>>>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>>>   . /run/current-system/profile/etc/profile
>>>   # [...]
>>>   . "$HOME/.guix-profile/etc/profile"
>>>   # [...]
>>>
>>> - /run/current-system/profile/etc/profile
>>>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>>>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>>>   # [...]
>>>
>>>
>>> - $HOME/.guix-profile/etc/profile
>>>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>>>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>>>   # [...]
>>>
>>> That is, each profile adds all its search paths to all the profile roots
>>> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
>>> is empty, it adds the search paths inside its own directory only (as
>>> until now).
>>
>> I think it was mentioned somewhere in the bug discussion: this will not
>> work for some things.  For example, if a user has 'guile' in a system
>> profile, and several guile packages (but not 'guile' itself) in
>> ~/.guix-profile, then GUILE_LOAD_PATH will include
>> "/share/guile/site/2.0" but not
>> "/share/guile/site/2.0".  That's why combining profiles
>> inside "guix package --search-paths" command looks like the only
>> solution.
>
> It will actually solve it, that is the case I was trying to fix in
> fact. The code I show will include /share/guile/site/2.0 prefixed
> by every root in the colon-separated list GUIX_PROFILES (note the
> difference with current code with has GUIX_PROFILE, singular). If
> GUIX_PROFILES is (as in the example):
>
> GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>
> And guile is in the system profile, the line
>
> export 
> GUILE_LOAD_PATH="${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"
>
> will produce:
>
> export 
> GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.0:$HOME/.guix-profile/share/guile/site/2.0

Unless I’m mistaken, I don’t think this would work.

In the example Alex gave, none of the two profiles specifies
GUILE_LOAD_PATH if taken individually.  This example is also in the
manual to illustrate the use of multiple -p flags with --search-paths
(info "(guix) Invoking guix package").

Thoughts?

Ludo’.

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Alex Kost  writes:

> Carlos Sánchez de La Lama (2016-09-20 09:39 +0200) wrote:
>
>> Hi,
>>
 Is this the intended behaviour? I am wondering whether packages with
 search paths should include both the user-profile directories and the
 system-profile ones.
>>>
>>> I think you’re right.  This was discussed at
>>> , leading to a patch (for GuixSD).
>>>
>>> However, we failed to build consensus around the approach of this patch,
>>> so we did not apply it.  If you have ideas, please email
>>> 20...@debbugs.gnu.org.  :-)
>>
>> what about something like:
>>
>> - /etc/profile
>>   # [...]
>>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>>   . /run/current-system/profile/etc/profile
>>   # [...]
>>   . "$HOME/.guix-profile/etc/profile"
>>   # [...]
>>
>> - /run/current-system/profile/etc/profile
>>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>>   # [...]
>>
>>
>> - $HOME/.guix-profile/etc/profile
>>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>>   # [...]
>>
>> That is, each profile adds all its search paths to all the profile roots
>> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
>> is empty, it adds the search paths inside its own directory only (as
>> until now).
>
> I think it was mentioned somewhere in the bug discussion: this will not
> work for some things.  For example, if a user has 'guile' in a system
> profile, and several guile packages (but not 'guile' itself) in
> ~/.guix-profile, then GUILE_LOAD_PATH will include
> "/share/guile/site/2.0" but not
> "/share/guile/site/2.0".  That's why combining profiles
> inside "guix package --search-paths" command looks like the only
> solution.

It will actually solve it, that is the case I was trying to fix in
fact. The code I show will include /share/guile/site/2.0 prefixed
by every root in the colon-separated list GUIX_PROFILES (note the
difference with current code with has GUIX_PROFILE, singular). If
GUIX_PROFILES is (as in the example):

GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"

And guile is in the system profile, the line

export 
GUILE_LOAD_PATH="${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"

will produce:

export 
GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.0:$HOME/.guix-profile/share/guile/site/2.0

As desired. Same result if guile is only in user profile, or if it is on
both. Try the following in bash:

profiles="/run/current-system/profile:$HOME/.guix-profile:/my/third/profile"
echo "${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"

BR

Carlos

Re: User-profile search paths should include system-profile directories

[Alex Kost]

Carlos Sánchez de La Lama (2016-09-20 09:39 +0200) wrote:

> Hi,
>
>>> Is this the intended behaviour? I am wondering whether packages with
>>> search paths should include both the user-profile directories and the
>>> system-profile ones.
>>
>> I think you’re right.  This was discussed at
>> , leading to a patch (for GuixSD).
>>
>> However, we failed to build consensus around the approach of this patch,
>> so we did not apply it.  If you have ideas, please email
>> 20...@debbugs.gnu.org.  :-)
>
> what about something like:
>
> - /etc/profile
>   # [...]
>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>   . /run/current-system/profile/etc/profile
>   # [...]
>   . "$HOME/.guix-profile/etc/profile"
>   # [...]
>
> - /run/current-system/profile/etc/profile
>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>   # [...]
>
>
> - $HOME/.guix-profile/etc/profile
>   profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>   # [...]
>
> That is, each profile adds all its search paths to all the profile roots
> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
> is empty, it adds the search paths inside its own directory only (as
> until now).

I think it was mentioned somewhere in the bug discussion: this will not
work for some things.  For example, if a user has 'guile' in a system
profile, and several guile packages (but not 'guile' itself) in
~/.guix-profile, then GUILE_LOAD_PATH will include
"/share/guile/site/2.0" but not
"/share/guile/site/2.0".  That's why combining profiles
inside "guix package --search-paths" command looks like the only
solution.

-- 
Alex

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi,

>> Is this the intended behaviour? I am wondering whether packages with
>> search paths should include both the user-profile directories and the
>> system-profile ones.
>
> I think you’re right.  This was discussed at
> , leading to a patch (for GuixSD).
>
> However, we failed to build consensus around the approach of this patch,
> so we did not apply it.  If you have ideas, please email
> 20...@debbugs.gnu.org.  :-)

what about something like:

- /etc/profile
  # [...]
  GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
  . /run/current-system/profile/etc/profile 
  # [...]
  . "$HOME/.guix-profile/etc/profile"
  # [...]

- /run/current-system/profile/etc/profile
  profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
  export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
  # [...]

  
- $HOME/.guix-profile/etc/profile
  profiles="${GUIX_PROFILES:=/gnu/store/-profile}"
  export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
  # [...]

That is, each profile adds all its search paths to all the profile roots
passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
is empty, it adds the search paths inside its own directory only (as
until now).

No need to expand recursively (no ":$OLD_VALUE" at then end of each
assignment) as we are generating the profile, user additions will come
later. Should be fast enough, and a user can do the (slower) guix
--search-paths in its own profile if he wants to.

If you think it's fine, I will prepare a patch.

Thoughts?

Carlos

Re: User-profile search paths should include system-profile directories

[Alex Kost]

Ludovic Courtès (2016-09-14 16:58 +0200) wrote:

> Hello,
>
> csanchez...@gmail.com (Carlos Sánchez de La Lama) skribis:
>
>> I have an interesting case here. I have guile installed in my system
>> profile, so that
>>
>> /var/guix/profiles/system/profile/share/aclocal/guile.m4
>>
>> is there. However, autoconf is installed in my *user* profile, so
>> ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include
>>
>> ~/.guix-profile/share/aclocal
>>
>> But not the system-profile aclocal directory, which would be put into
>> ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
>> autoconf was installed in the systme profile as well.
>>
>> Is this the intended behaviour? I am wondering whether packages with
>> search paths should include both the user-profile directories and the
>> system-profile ones.
>
> I think you’re right.  This was discussed at
> , leading to a patch (for GuixSD).
>
> However, we failed to build consensus around the approach of this patch,
> so we did not apply it.  If you have ideas, please email
> 20...@debbugs.gnu.org.  :-)

I think I was the one who prevents the consensus.  To make it clear, I'm
for the suggested solution, but only *after* giving a user a freedom to
avoid loading such a heavy command as "guix package --search-paths".  On
a "usual" GNU/Linux distro a user can edit /etc/profile, but on GuixSD
it is not possible currently.  That's why I think there should be
provided a possibility to override /etc/profile at first.

> One of the fruits of this discussion, though, is that you can do:
>
>   guix package -p /run/current-system/profile -p ~/.guix-profile 
> --search-paths
>
> to get the combined search paths.

This is what I do to combine my profiles (I don't use ~/.guix-profile
but several other profiles instead).

-- 
Alex

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hello,

csanchez...@gmail.com (Carlos Sánchez de La Lama) skribis:

> I have an interesting case here. I have guile installed in my system
> profile, so that
>
> /var/guix/profiles/system/profile/share/aclocal/guile.m4
>
> is there. However, autoconf is installed in my *user* profile, so
> ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include
>
> ~/.guix-profile/share/aclocal
>
> But not the system-profile aclocal directory, which would be put into
> ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
> autoconf was installed in the systme profile as well.
>
> Is this the intended behaviour? I am wondering whether packages with
> search paths should include both the user-profile directories and the
> system-profile ones.

I think you’re right.  This was discussed at
, leading to a patch (for GuixSD).

However, we failed to build consensus around the approach of this patch,
so we did not apply it.  If you have ideas, please email
20...@debbugs.gnu.org.  :-)

One of the fruits of this discussion, though, is that you can do:

  guix package -p /run/current-system/profile -p ~/.guix-profile --search-paths

to get the combined search paths.

Ludo’.