Re: [Nix-dev] break purity

[stewart mackenzie]

Hi Oliver,

Here is a minimal example stripped of all the other crap:

https://github.com/fractalide/recompilation

Cheers!

/sjm
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Oliver Charles]

I'm afraid I don't really know what I'm looking at. Can you provide a
smaller self-contained example that I can look at?

On Sat, Apr 23, 2016 at 8:48 PM stewart mackenzie 
wrote:

> Okay, I'm having a few problems implementing this, if you wouldn't
> mind taking a look at this please:
>
> This is the package level default.nix which calls
> `buildFractalideComponent`: http://nixpaste.lbr.uno/ZTwRzV0-?nix
> Typically found here:
>
> https://github.com/fractalide/fractalide/blob/master/components/drop/ip/default.nix
>
> Here is the implementation of `buildFractalideComponent.nix`:
> http://nixpaste.lbr.uno/GmbsNjmk?nix
> Found here:
> https://github.com/fractalide/fractalide/blob/master/build-support/buildFractalideComponent.nix
>
> Here's my error message:
>
> [stewart@rivergod:~/dev/fractalide/fractalide]$ nix-build --argstr
> debug true -A components.drop_ip
> error: value is a function while a set was expected, at
>
> /home/stewart/dev/fractalide/fractalide/components/drop/ip/default.nix:15:21
> (use ‘--show-trace’ to show detailed location information)
>
> The error arises on the last line of default.nix
>
> /sjm
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

Okay, I'm having a few problems implementing this, if you wouldn't
mind taking a look at this please:

This is the package level default.nix which calls
`buildFractalideComponent`: http://nixpaste.lbr.uno/ZTwRzV0-?nix
Typically found here:
https://github.com/fractalide/fractalide/blob/master/components/drop/ip/default.nix

Here is the implementation of `buildFractalideComponent.nix`:
http://nixpaste.lbr.uno/GmbsNjmk?nix
Found here: 
https://github.com/fractalide/fractalide/blob/master/build-support/buildFractalideComponent.nix

Here's my error message:

[stewart@rivergod:~/dev/fractalide/fractalide]$ nix-build --argstr
debug true -A components.drop_ip
error: value is a function while a set was expected, at
/home/stewart/dev/fractalide/fractalide/components/drop/ip/default.nix:15:21
(use ‘--show-trace’ to show detailed location information)

The error arises on the last line of default.nix

/sjm
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

Oliver, what is in prelude/shell.nix? (nix-build -E '(import
prelude/shell.nix { nixpkgs = import /home/ollie/nixpkgs {}; }).dist')

/sjm
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Profpatsch]

On 16-04-22 06:33pm, Oliver Charles wrote:
> It's certainly possible, because I just did it with Haskell ;)
> 
> https://gist.github.com/ocharles/cbd5d7ce63bb570abb86e655f36435ab

Why do you need to put the atime two hours in the past?
Does cabal recompilation check for it in order to find out
what changed?


-- 
Proudly written in Mutt with Vim on NixOS.
Q: Why is this email five sentences or less?
A: http://five.sentenc.es
May take up to five days to read your message. If it’s urgent, call me.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Vladimír Čunát]

A completely different approach would be to somehow break up your big
build into multiple derivations. This thread reminded me of nix-make:
https://github.com/edolstra/nix-make/blob/master/examples/not-so-simple-header-auto/default.nix

--Vladimir




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Vladimír Čunát]

On 04/22/2016 08:33 PM, Oliver Charles wrote:
> It's certainly possible, because I just did it with Haskell ;)
> 
> https://gist.github.com/ocharles/cbd5d7ce63bb570abb86e655f36435ab

Hehe, the classic Haskell approach: make the impurity explicit in order
not to actually break purity...




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Aycan iRiCAN]

This is a clever optimization. Thank you Oliver.

--
aycan

> On 22 Apr 2016, at 21:33, Oliver Charles  wrote:
> 
> It's certainly possible, because I just did it with Haskell ;)
> 
> https://gist.github.com/ocharles/cbd5d7ce63bb570abb86e655f36435ab
> 
>> On Fri, Apr 22, 2016 at 7:16 PM stewart mackenzie  wrote:
>> Yeah, I'm not so sure it's possible because one cannot copy from a
>> precompiled derivation output to a new derivation output, ie copy
>> cross derivation.
>> 
>> As these multiple outputs (outputs = [x y z]) are seen as different
>> derivations, this cannot happen right?
>> 
>> Have you actually managed to make progress beyond the copying part of
>> your example (preConfigure = "cp ${artefacts}/* .'';) in your
>> experimentation?
>> 
>> (I'm still thought experimenting)
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

You will save me hours of waiting in the future, definitely beer worthy!

Much appreciated Oliver!
On 23 Apr 2016 02:33, "Oliver Charles"  wrote:

> It's certainly possible, because I just did it with Haskell ;)
>
> https://gist.github.com/ocharles/cbd5d7ce63bb570abb86e655f36435ab
>
> On Fri, Apr 22, 2016 at 7:16 PM stewart mackenzie 
> wrote:
>
>> Yeah, I'm not so sure it's possible because one cannot copy from a
>> precompiled derivation output to a new derivation output, ie copy
>> cross derivation.
>>
>> As these multiple outputs (outputs = [x y z]) are seen as different
>> derivations, this cannot happen right?
>>
>> Have you actually managed to make progress beyond the copying part of
>> your example (preConfigure = "cp ${artefacts}/* .'';) in your
>> experimentation?
>>
>> (I'm still thought experimenting)
>>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Oliver Charles]

It's certainly possible, because I just did it with Haskell ;)

https://gist.github.com/ocharles/cbd5d7ce63bb570abb86e655f36435ab

On Fri, Apr 22, 2016 at 7:16 PM stewart mackenzie 
wrote:

> Yeah, I'm not so sure it's possible because one cannot copy from a
> precompiled derivation output to a new derivation output, ie copy
> cross derivation.
>
> As these multiple outputs (outputs = [x y z]) are seen as different
> derivations, this cannot happen right?
>
> Have you actually managed to make progress beyond the copying part of
> your example (preConfigure = "cp ${artefacts}/* .'';) in your
> experimentation?
>
> (I'm still thought experimenting)
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

Yeah, I'm not so sure it's possible because one cannot copy from a
precompiled derivation output to a new derivation output, ie copy
cross derivation.

As these multiple outputs (outputs = [x y z]) are seen as different
derivations, this cannot happen right?

Have you actually managed to make progress beyond the copying part of
your example (preConfigure = "cp ${artefacts}/* .'';) in your
experimentation?

(I'm still thought experimenting)
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

Ah, okay, I get your drift, I'll think how to make it succinct and
tidy in the code :-)
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Oliver Charles]

Well you don't tie the knot in the expression itself. It's more that the
pre-existing artefacts are a function input, and a function output:

{ stdenv, gcc, etc, artefacts ? null }:
{
  preConfigure = "cp ${artefacts}/* .'';
  outputs = [ "out" "artefacts" ];
}

Then you would nix-build to produce binaries and an artefacts directory.
Subsequent nix-builds would then use nix-build --arg 'artefacts'
'./results-artefacts'.

Handwaving, but hopefully that helps clarify.
ocharles

On Fri, Apr 22, 2016 at 5:35 PM stewart mackenzie 
wrote:

> Interesting Oliver, though this sounds like it'll throw a recursive
> error. I'll investigate! cheers!
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[stewart mackenzie]

Interesting Oliver, though this sounds like it'll throw a recursive
error. I'll investigate! cheers!
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Oliver Charles]

Something I'm experimenting with here is to actually have these
intermediate artefacts be explicitly captured. I would use multiple outputs
to have both the normal complete binaries, and another output for .o files
and the like. Then you can feed this back in to future builds.

Does that help? Might give you some food for thought...
ocharles

On Thu, 21 Apr 2016, 4:42 p.m. stewart mackenzie, 
wrote:

> Hi,
>
> I've got a bunch of artifacts being built in the
> /tmp/nix-build-component_name.drv-0
>
> now I want to point the build manager, in this case cargo using the
> CARGO_TARGET_DIR env var to put the artifacts into a directory I make
> directly in the /tmp/target_${name}
>
> This way any further compilations will reuse the artifacts and they
> won't need to be rebuilt.
>
> obviously nix doesn't allow me to do this.
>
> Is there a dirty hack to allow creating and writing artifacts directly
> to /tmp/target_${name}?
>
> Why? We're using nix as a replacement for make, one major drawback is:
> every time we change a line of code the entire component + trans deps
> needs to be rebuilt. This takes a long time when transitive deps go
> deep.
>
> /sjm
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] break purity

[stewart mackenzie]

Hi,

I've got a bunch of artifacts being built in the
/tmp/nix-build-component_name.drv-0

now I want to point the build manager, in this case cargo using the
CARGO_TARGET_DIR env var to put the artifacts into a directory I make
directly in the /tmp/target_${name}

This way any further compilations will reuse the artifacts and they
won't need to be rebuilt.

obviously nix doesn't allow me to do this.

Is there a dirty hack to allow creating and writing artifacts directly
to /tmp/target_${name}?

Why? We're using nix as a replacement for make, one major drawback is:
every time we change a line of code the entire component + trans deps
needs to be rebuilt. This takes a long time when transitive deps go
deep.

/sjm
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] break purity

[Teo Klestrup Röijezon]

You can set up a build environment resembling what nix-build would use
using the nix-shell command. All dependencies are provided, and each build
phase gets a function.

On 21 April 2016 at 17:42, stewart mackenzie  wrote:

> Hi,
>
> I've got a bunch of artifacts being built in the
> /tmp/nix-build-component_name.drv-0
>
> now I want to point the build manager, in this case cargo using the
> CARGO_TARGET_DIR env var to put the artifacts into a directory I make
> directly in the /tmp/target_${name}
>
> This way any further compilations will reuse the artifacts and they
> won't need to be rebuilt.
>
> obviously nix doesn't allow me to do this.
>
> Is there a dirty hack to allow creating and writing artifacts directly
> to /tmp/target_${name}?
>
> Why? We're using nix as a replacement for make, one major drawback is:
> every time we change a line of code the entire component + trans deps
> needs to be rebuilt. This takes a long time when transitive deps go
> deep.
>
> /sjm
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev