Re: Package input loop detection

[Ricardo Wurmus]

Ricardo Wurmus  writes:

>> I've had this issue for a while now, while adding some packages, I'll
>> create a loop in the package graph, which causes Guix to just loop
>> infinitely when trying to generate derivations.
>
> this is a great initiative.  I’ve been having this issue in the past as
> well, and I’d really like Guix to be a little smarter about it.

I’m currently updating many Haskell packages and I applied this patch to
make this job easier.  It works as advertised.

I think it could be less verbose (because something like this won’t
happen in a released version of Guix and is only useful to developers).
I’m no longer convinced that it should be a separate tool; it is an
improvement over having “guix build” hang indefinitely.  I’d much rather
have “guix build” fail on its own than requiring me to lose patience and
hit Ctrl-C.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Package input loop detection

[Gábor Boskovits]

2018-02-12 16:30 GMT+01:00 Ricardo Wurmus :

>
> Hi,
>
> > I've had this issue for a while now, while adding some packages, I'll
> > create a loop in the package graph, which causes Guix to just loop
> > infinitely when trying to generate derivations.
>
> this is a great initiative.  I’ve been having this issue in the past as
> well, and I’d really like Guix to be a little smarter about it.
>
>
I also welcome this idea.


> > I'm not particularly fond of the implementation, because the
> > package-derivation function is called from expand-input called from
> > bag->derivation, the information about the part of the graph that has
> > been traversed is passed through each function.
> >
> > The seen-package-list argument could be removed, but the ordering
> > information is really useful when printing out the error message. I
> > think it should be still possible to generate this after finding the
> > issue by searching through the graph of packages, which would allow
> > removing this one argument.
> >
> > One other thought I had is that this could be extracted to something in
> > guix lint, which would at least allow finding these problems, without
> > touching the core derivation related code.
>
> I’d be in favour of keeping it out of the core and stash it away in a
> separate tool.  Not sure if that should be “guix lint” (what about “guix
> graph”?), but I would prefer that over having the code run all the time.
>
>
I feel that "guix graph" is a good suggestion.
Correct me if I'm mistaken, but it seems to me that we currently traverse
our graph breadth first. For this functionality a depth first traversal
would have
benefits. Could be extended to topologically short the whole DAG. WDYT?


> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>
>

Re: Passing substitute-urls via GUIX_BUILD_OPTIONS

[Martin Castillo]

On 12.02.2018 11:53, Marco van Zwetselaar wrote:
> All,
> 
> Is there a magic escape to allow passing --substitute-urls via
> GUIX_BUILD_OPTIONS?
> 
Not that I know of. Probably not.

> 
> I can't seem to find the incantation that works.
> 

This is known as bug #29814.

Martin Castillo

-- 
GPG: 7FDE 7190 2F73 2C50 236E  403D CC13 48F1 E644 08EC

Re: Package input loop detection

[Ricardo Wurmus]

Hi,

> I've had this issue for a while now, while adding some packages, I'll
> create a loop in the package graph, which causes Guix to just loop
> infinitely when trying to generate derivations.

this is a great initiative.  I’ve been having this issue in the past as
well, and I’d really like Guix to be a little smarter about it.

> I'm not particularly fond of the implementation, because the
> package-derivation function is called from expand-input called from
> bag->derivation, the information about the part of the graph that has
> been traversed is passed through each function.
>
> The seen-package-list argument could be removed, but the ordering
> information is really useful when printing out the error message. I
> think it should be still possible to generate this after finding the
> issue by searching through the graph of packages, which would allow
> removing this one argument.
>
> One other thought I had is that this could be extracted to something in
> guix lint, which would at least allow finding these problems, without
> touching the core derivation related code.

I’d be in favour of keeping it out of the core and stash it away in a
separate tool.  Not sure if that should be “guix lint” (what about “guix
graph”?), but I would prefer that over having the code run all the time.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Passing substitute-urls via GUIX_BUILD_OPTIONS

[Ricardo Wurmus]

Hi Marco,

> Is there a magic escape to allow passing --substitute-urls via
> GUIX_BUILD_OPTIONS?

Must it be done via GUIX_BUILD_OPTIONS?  The daemon can be started with
that option and it will be used for all invocations of the “guix”
command.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: proposal: modularize unpack in gnu build system

[Gábor Boskovits]

2018-02-11 22:50 GMT+01:00 Ricardo Wurmus :

>
> Gábor Boskovits  writes:
>
> > It seems, that currently we don't have zip in gnu build system, so that
> > ends up in native-inputs, if we need that to unpack the tarball.
> >
> > It would be nice if gnu-build-system would use the appropriate
> compressor,
> > and at the same time only depend on the one needed. How could this be
> > achieved?
>
> Why is this necessary?  I don’t see a problem with adding “unzip” to the
> native-inputs when it is needed to unpack a zip archive.
>
> In my opinion this is so easy that I don’t think the slight increase in
> convenience would be worth the change.
>
>
You might be right about that. What I feel is that:
1. this is a recurring newbie question on the irc channel, I've seen it
already a few times, so support time could be reduced
2. this change could make adding new archive types easier in the future.


> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>

Re: proposal: modularize unpack in gnu build system

[Ricardo Wurmus]

Gábor Boskovits  writes:

> It seems, that currently we don't have zip in gnu build system, so that
> ends up in native-inputs, if we need that to unpack the tarball.
>
> It would be nice if gnu-build-system would use the appropriate compressor,
> and at the same time only depend on the one needed. How could this be
> achieved?

Why is this necessary?  I don’t see a problem with adding “unzip” to the
native-inputs when it is needed to unpack a zip archive.

In my opinion this is so easy that I don’t think the slight increase in
convenience would be worth the change.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Use guix to distribute data & reproducible (data) science

[Konrad Hinsen]

Hi everyone,

zimoun  writes:

> From my point of view, there is 2 kind of datasets:
>  a- the ones which are part of the software, e.g., used to pass the
> tests. Therefore, they are usually small, not always;
>  b- the ones which are applied to the software and somehow they are
> not in the source repository. They are big or not.

I was thinking of the second kind only.

> It does not appear to me a good idea to try to include in the store
> datasets of case b-.
> Is it not the job of data management tools ? e.g., database etc.

It depends. The frontier between data and code is not as clear as it may
seem. An example: the weights of a trained neural network can be seen as
data (a bunch of numbers), but also as code for a special-purpose
processor defined by the neural network.

Starting from that example, consider that the weights of a neural
network are not fundamentally different from fit parameters in other
scientific models. For example the positions of the atoms in a protein
structure. Using the same analogy as for the neural network, these
positions are the code for a special-purpose processor that computes
estimations for the Bragg reflections that are measured in protein
crystallography.

On the other hand, any sufficiently small dataset can be replaced by a
piece of code that defines a literal data structure. Does that turn the
data into code or not?

My point is that trying to define a distinction between data and code is
always arbitrary and rarely of practical interest. I prefer to take a
pragmatic stance and ask the question: what are the advantages and
problems associated with managing some piece of data in the store? And I
suspect that exploring this question for a couple of applications will
lead to new ways to profit from the store.

However, distribution of data is an entirely different question from
managing data on a single machine. I have no idea how well suited the
store is for distributing data (Hydra does exactly that, right?), so
I'll happily listen to the Guix experts.

> It appear to me as a complement of these points ---and personnally, I
> learn some points about the design of GWL--- with this thread:
> https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00380.html

Thanks for the pointer!

>> It would be nice if big datasets could conceptually be handled in the same
>> way while being stored elsewhere - a bit like git-annex does for git. And
>> for parallel computing, we could have special build daemons.
>
> Hum? the point is to add data management a la git-annex to GWL ? Is it ?

At least consider it - I don't know where that will lead.


Amirouche Boubekki  writes:

>> For big datasets, some other mechanism is required.
>
> Big as in bigger than ram?

Bigger that one is willing to handle as files that are copied between
machines for distribution. For me, personally and at this moment in
time, that's somewhere between 1 GB and 10 GB.

> What I was thinking about, is use guix to distribute data packages
> just like we distribute softwares from pypi. The advantage of using
> guix seems obvious, but apparantly it's not desirable or possible and
> I don't understand why.

I think there are two questions:
 1. Can guix and its current infrastructure be used for data
 distribution?
 2. Can a yet-to-be-designed data distribution infrastructure
 use the guix store as its storage backend?

My understanding from Ludo's comment is that 1) may not be a good idea,
but that still leaves 2) as a topic for exploration.

> And for parallel computing, we could have special build daemons.
>
> That's where OWL comes in?

Exactly.

Konrad.

Passing substitute-urls via GUIX_BUILD_OPTIONS

[Marco van Zwetselaar]

All,

Is there a magic escape to allow passing --substitute-urls via
GUIX_BUILD_OPTIONS?

$ GUIX_BUILD_OPTIONS="--substitute-urls=http://server1
https://mirror.hydra.gnu.org; guix package -I
guix package: error: https://mirror.hydra.gnu.org: extraneous argument
$ GUIX_BUILD_OPTIONS="--substitute-urls='http://server1
https://mirror.hydra.gnu.org'" guix package -I
guix package: error: https://mirror.hydra.gnu.org': extraneous argument
$ GUIX_BUILD_OPTIONS='--substitute-urls="http://server1
https://mirror.hydra.gnu.org;' guix package -I
guix package: error: https://mirror.hydra.gnu.org": extraneous argument
$ ... other attempts involving apostrophes, quotes, and backslashes ...

I can't seem to find the incantation that works.

Cheers,
Marco

New Chinese (simplified) PO file for 'guix' (version 0.14.0)

[Translation Project Robot]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'guix' has been submitted
by the Chinese (simplified) team of translators.  The file is available at:

http://translationproject.org/latest/guix/zh_CN.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

http://translationproject.org/latest/guix/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

http://translationproject.org/domain/guix.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

The Translation Project robot, in the
name of your translation coordinator.

Re: FOSDEM 2018 and announcing a GNU Guix/Guile day! After getting home...

[Efraim Flashner]

On Mon, Feb 05, 2018 at 12:23:37PM +0100, Alex Sassmannshausen wrote:
> Hello,
> 
> It sounds like the event was absolutely amazing — I'm well jealous of
> everyone who was able to go!
> 
> Ludovic Courtès writes:
> 
> > That alone could justify pursuing the same approach in the coming
> > years, IMO.  At the same time, a devroom at FOSDEM proper is a good
> > way to reach out to new people, which is also a good thing.  Maybe we
> > could do both?  :-)
> 
> Yes please! That would give me another chance to participate :D
> 
> Alex
> 

The last time I looked at booths for FOSDEM they said it was really
crowded and that even groups that might have previously gotten two
tables would only be getting one table. For next year we might consider
pairing up with GNU or with Nix and splitting a table. It'd give us a
meeting area to hang around also.

-- 
Efraim Flashner      אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature