Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 27 September 2014 11:12, Vladimír Čunát vcu...@gmail.com wrote:
On 09/24/2014 09:58 PM, Bjørn Forsman wrote:
On 22 September 2014 17:16, Shea Levy s...@shealevy.com wrote:
I suppose a flag that says to rebuild if the output is valid but doesn't
have the right deriver could be what you want? But for now the best
answer
seems to be change the hash when it needs to change
I don't know what deriver is.
Deriver is the derivation that built a path (i.e. the build recipe).
If you built a path yourself, you can get it by
nix-store -q --deriver /nix/store/path
Ah, thanks.
Shea, it looks like the deriver (*.drv) doesn't change unless name
or the output hash attrs changes. So rebuild-if-deriver-changed won't
make any difference.
Even though it's a bit hackish, using
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
works just like I want it to.
Ouch, it broke real bad when using it in the real world[1]:
fetchGrailsAppDeps = { stdenv, grails, src, sha256 }:
stdenv.mkDerivation rec {
srcHash = builtins.substring 0 20 (builtins.hashString sha256
(builtins.toString src));
name = grails-app-deps-${srcHash};
[SNIP]
};
$ nix-build grails-app.nix
error: the string `grails-app-deps-b0f29513403e6dca5372' is not
allowed to refer to a store path (such as
`!out!/nix/store/80bd47sghjl4i41sd783b2w49lqzkm93-git-export.drv'), at
/nix/store/rjvnj4skg17is5x9jhzq1yl96mdvch0c-nix-1.7/share/nix/corepkgs/derivation.nix:8:12
This only happens when src = fetchgit { ... } and not when src =
./path/to/source. Why?
And what does this check protect against?
Now I have a big fat warning IMPORTANT: Remember to force re-download
deps whenever you update the source! in my expression :-(
Oh, this is bad...
I think I understand what you want: the output should not change except on
updating the package itself, but one can only find out the hash by
performing the build. On update you want to do that.
I personally force such rebuilds by changing the first few sha256 characters
to zeros. The extraHash way and Shea's suggestion are other fine ways. For
this use case, I find no clear winner, and I don't think we can do
significantly better.
What about a new, optional derivation input like extraHashInputs = [
list of attrs that will be included in calculation of $out ];?
[1]: https://gist.github.com/bjornfor/3f2fda1b1ce077225df5
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On Wed, Oct 01, 2014 at 10:32:57AM +0200, Bjørn Forsman wrote:
On 27 September 2014 11:12, Vladimír Čunát vcu...@gmail.com wrote:
On 09/24/2014 09:58 PM, Bjørn Forsman wrote:
On 22 September 2014 17:16, Shea Levy s...@shealevy.com wrote:
I suppose a flag that says to rebuild if the output is valid but doesn't
have the right deriver could be what you want? But for now the best
answer
seems to be change the hash when it needs to change
I don't know what deriver is.
Deriver is the derivation that built a path (i.e. the build recipe).
If you built a path yourself, you can get it by
nix-store -q --deriver /nix/store/path
Ah, thanks.
Shea, it looks like the deriver (*.drv) doesn't change unless name
or the output hash attrs changes. So rebuild-if-deriver-changed won't
make any difference.
Nope, deriver will change if any attribute changes. Or can you show me
an example otherwise?
Even though it's a bit hackish, using
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
works just like I want it to.
Ouch, it broke real bad when using it in the real world[1]:
fetchGrailsAppDeps = { stdenv, grails, src, sha256 }:
stdenv.mkDerivation rec {
srcHash = builtins.substring 0 20 (builtins.hashString sha256
(builtins.toString src));
name = grails-app-deps-${srcHash};
[SNIP]
};
$ nix-build grails-app.nix
error: the string `grails-app-deps-b0f29513403e6dca5372' is not
allowed to refer to a store path (such as
`!out!/nix/store/80bd47sghjl4i41sd783b2w49lqzkm93-git-export.drv'), at
/nix/store/rjvnj4skg17is5x9jhzq1yl96mdvch0c-nix-1.7/share/nix/corepkgs/derivation.nix:8:12
This only happens when src = fetchgit { ... } and not when src =
./path/to/source. Why?
Use builtins.unsafeDiscardStringContext on the result of toString src
before hashString.
And what does this check protect against?
Now I have a big fat warning IMPORTANT: Remember to force re-download
deps whenever you update the source! in my expression :-(
Oh, this is bad...
I think I understand what you want: the output should not change except on
updating the package itself, but one can only find out the hash by
performing the build. On update you want to do that.
I personally force such rebuilds by changing the first few sha256 characters
to zeros. The extraHash way and Shea's suggestion are other fine ways. For
this use case, I find no clear winner, and I don't think we can do
significantly better.
What about a new, optional derivation input like extraHashInputs = [
list of attrs that will be included in calculation of $out ];?
[1]: https://gist.github.com/bjornfor/3f2fda1b1ce077225df5
Please no. This is fixing the problem in the wrong place.
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
Hi Shea,
On 1 October 2014 17:59, Shea Levy s...@shealevy.com wrote:
On Wed, Oct 01, 2014 at 10:32:57AM +0200, Bjørn Forsman wrote:
Shea, it looks like the deriver (*.drv) doesn't change unless name
or the output hash attrs changes. So rebuild-if-deriver-changed won't
make any difference.
Nope, deriver will change if any attribute changes. Or can you show me
an example otherwise?
Oh, you're right. It does change. I tested it in nix-repl (perfect
tool for the job!). Earlier today I didn't use nix-repl and must have
messed up when calling nix-build and nix-store -q --deriver.
So yes, a rebuild-if-deriver-changed option would be nice.
Even though it's a bit hackish, using
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
works just like I want it to.
Ouch, it broke real bad when using it in the real world[1]:
fetchGrailsAppDeps = { stdenv, grails, src, sha256 }:
stdenv.mkDerivation rec {
srcHash = builtins.substring 0 20 (builtins.hashString sha256
(builtins.toString src));
name = grails-app-deps-${srcHash};
[SNIP]
};
$ nix-build grails-app.nix
error: the string `grails-app-deps-b0f29513403e6dca5372' is not
allowed to refer to a store path (such as
`!out!/nix/store/80bd47sghjl4i41sd783b2w49lqzkm93-git-export.drv'), at
/nix/store/rjvnj4skg17is5x9jhzq1yl96mdvch0c-nix-1.7/share/nix/corepkgs/derivation.nix:8:12
This only happens when src = fetchgit { ... } and not when src =
./path/to/source. Why?
Use builtins.unsafeDiscardStringContext on the result of toString src
before hashString.
Thank you very much! Now the world makes sense again.
What about a new, optional derivation input like extraHashInputs = [
list of attrs that will be included in calculation of $out ];?
[1]: https://gist.github.com/bjornfor/3f2fda1b1ce077225df5
Please no. This is fixing the problem in the wrong place.
Ok. But a rebuildIfDeriverChanged = true; flag then?
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 10/01/2014 09:15 PM, Bjørn Forsman wrote: Ok. But a rebuildIfDeriverChanged = true; flag then? A nix-build option for that would make much more sense to me than an argument to mkDerivation. (IIRC that's what Shea intended originally.) Vladimir smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 1 October 2014 21:33, Vladimír Čunát vcu...@gmail.com wrote: On 10/01/2014 09:15 PM, Bjørn Forsman wrote: Ok. But a rebuildIfDeriverChanged = true; flag then? A nix-build option for that would make much more sense to me than an argument to mkDerivation. (IIRC that's what Shea intended originally.) Ok. My first thought was that a nix-build flag would be a very big hammer, and that it could cause unneeded rebuilds. But maybe you're right. Best regards, Bjørn Forsman ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 10/01/2014 09:50 PM, Bjørn Forsman wrote: My first thought was that a nix-build flag would be a very big hammer, and that it could cause unneeded rebuilds. Hmm, with the flag you would have rebuilds of e.g. all things where you changed the download URI, true. But that seems an unlikely combination of events. smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 09/24/2014 09:58 PM, Bjørn Forsman wrote:
On 22 September 2014 17:16, Shea Levy s...@shealevy.com wrote:
I suppose a flag that says to rebuild if the output is valid but doesn't
have the right deriver could be what you want? But for now the best answer
seems to be change the hash when it needs to change
I don't know what deriver is.
Deriver is the derivation that built a path (i.e. the build recipe).
If you built a path yourself, you can get it by
nix-store -q --deriver /nix/store/path
Even though it's a bit hackish, using
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
works just like I want it to.
I think I understand what you want: the output should not change except
on updating the package itself, but one can only find out the hash by
performing the build. On update you want to do that.
I personally force such rebuilds by changing the first few sha256
characters to zeros. The extraHash way and Shea's suggestion are other
fine ways. For this use case, I find no clear winner, and I don't think
we can do significantly better.
Vladimir
smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 22 September 2014 11:52, Wout Mertens wout.mert...@gmail.com wrote: Isn't the whole point of the fixed-output derivations that the means of getting there doesn't matter? So if you change the build script but leave the output hash the same, you're basically saying that the build script will generate the same output. Right. So I guess what I want isn't really a fixed-output derivation, but more of a checked-output derivation. In my use case, I don't have direct control over buildCommand. I run grails -Duser.home=$out refresh-dependencies in the source tree and grails downloads a bunch of stuff according to what's specified in $src/.../BuildConfig.groovy. (I've written a fetchGrailsAppDeps function for that.) 1) The derivation must be fixed-output to allow internet access 2) The output isn't _really_ fixed, because it can change if $src changes (new application source may have different deps) I think what you want is to remove a build product from the store when you change the build scripts? That feels hackish and not something I want to be doing on a regular basis. As for this use case, handbrake also downloads a bunch of things as part of the build and I made the expression just patch that away, moving all the extra building and patches into other expressions. I considered doing that. But the solution / workaround by adding the source hash to derivation name (to force rebuild if changed) works so well :-) Best regards, Bjørn Forsman ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 22 September 2014 17:16, Shea Levy s...@shealevy.com wrote:
I suppose a flag that says to rebuild if the output is valid but doesn't
have the right deriver could be what you want? But for now the best answer
seems to be change the hash when it needs to change
I don't know what deriver is.
Even though it's a bit hackish, using
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
works just like I want it to.
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
Isn't the whole point of the fixed-output derivations that the means of
getting there doesn't matter? So if you change the build script but leave
the output hash the same, you're basically saying that the build script
will generate the same output.
I think what you want is to remove a build product from the store when you
change the build scripts?
As for this use case, handbrake also downloads a bunch of things as part of
the build and I made the expression just patch that away, moving all the
extra building and patches into other expressions.
Another approach might be to use recursive Nix (building new nix
expressions as part of a nix build), although I'm not sure what the state
of that is.
Wout.
On Sun, Sep 21, 2014 at 6:37 PM, Bjørn Forsman bjorn.fors...@gmail.com
wrote:
On 21 September 2014 08:15, Vladimír Čunát vcu...@gmail.com wrote:
On 09/20/2014 09:55 PM, Bjørn Forsman wrote:
Since fetchurl (via the nixos tarball mirror) downloads the same old
file (even though it creates a new store path), the result outputHash
is the same as before. So evaluation doesn't stop but continues on
with old/wrong data.
If fetchurl *did* download any other data, then it would fail while
checking
the hash. That was my point. Nixos tarball mirror may be different and
download the same content, but in any case that doesn't change the plain
fact of the first sentence of this paragraph.
I think we all agree on how fetchurl works, but I still have no idea what
you desire. I may be an exception, but I don't see what could be done
between the current fixed-output and standard derivations. Either you
do
know what exactly the output should look like and then it doesn't really
matter how you arrive at it, or you don't know it and then you have a
standard derivation.
I'll try to explain the issue with an example. Given the following
expression:
with (import nixpkgs { });
stdenv.mkDerivation rec {
name = foo-0.0;
outputHashAlgo = sha256;
outputHash = 0sldaa6dd6dsfm1c7f8z9i32dfhn53aw1kyrlid1naa7hb6biy4v;
outputHashMode = recursive;
buildCommand = ''
mkdir -p $out
echo hello $out/file
'';
}
Build it. Then change the echo hello $out/file to echo foobar
$out/file. The output will clearly change, but nix has cached the
result and will not do a rebuild. That's bad!
(You may at this point wonder, why use fixed-output derivation at all?
It's because the buildCommand, as is the case when packaging grails
apps, will download dependencies from the internet, so a standard
derivation will fail.)
I can work around this caching this by adding some hackery:
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
This results in a derivation that gets rebuilt/checked whenever
buildCommand changes. This is what I want.
But this solution feels very hackish. So my question is whether it's
possible to tell nix that certain derivation inputs affect the output,
so that when the inputs change, nix has to rebuild / re-check the
result.
It seems that when using fixed-output derivations, only the name
attribute matters. I think there are use cases (like the example
above) that requires other derivation inputs to be considered
important enough to do a rebuild.
Vladimir, does it make sense to you now?
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
I suppose a flag that says to rebuild if the output is valid but doesn't have
the right deriver could be what you want? But for now the best answer seems to
be change the hash when it needs to change
div Original message /divdivFrom: Bjørn Forsman
bjorn.fors...@gmail.com /divdivDate:09/21/2014 9:37 AM (GMT-08:00)
/divdivTo: Vladimír Čunát vcu...@gmail.com /divdivCc: nix-dev
nix-dev@lists.science.uu.nl /divdivSubject: Re: [Nix-dev] fixed-output
derivation that *also* depend on (some
of) its inputs? /divdiv
/divOn 21 September 2014 08:15, Vladimír Čunát vcu...@gmail.com wrote:
On 09/20/2014 09:55 PM, Bjørn Forsman wrote:
Since fetchurl (via the nixos tarball mirror) downloads the same old
file (even though it creates a new store path), the result outputHash
is the same as before. So evaluation doesn't stop but continues on
with old/wrong data.
If fetchurl *did* download any other data, then it would fail while checking
the hash. That was my point. Nixos tarball mirror may be different and
download the same content, but in any case that doesn't change the plain
fact of the first sentence of this paragraph.
I think we all agree on how fetchurl works, but I still have no idea what
you desire. I may be an exception, but I don't see what could be done
between the current fixed-output and standard derivations. Either you do
know what exactly the output should look like and then it doesn't really
matter how you arrive at it, or you don't know it and then you have a
standard derivation.
I'll try to explain the issue with an example. Given the following expression:
with (import nixpkgs { });
stdenv.mkDerivation rec {
name = foo-0.0;
outputHashAlgo = sha256;
outputHash = 0sldaa6dd6dsfm1c7f8z9i32dfhn53aw1kyrlid1naa7hb6biy4v;
outputHashMode = recursive;
buildCommand = ''
mkdir -p $out
echo hello $out/file
'';
}
Build it. Then change the echo hello $out/file to echo foobar
$out/file. The output will clearly change, but nix has cached the
result and will not do a rebuild. That's bad!
(You may at this point wonder, why use fixed-output derivation at all?
It's because the buildCommand, as is the case when packaging grails
apps, will download dependencies from the internet, so a standard
derivation will fail.)
I can work around this caching this by adding some hackery:
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
This results in a derivation that gets rebuilt/checked whenever
buildCommand changes. This is what I want.
But this solution feels very hackish. So my question is whether it's
possible to tell nix that certain derivation inputs affect the output,
so that when the inputs change, nix has to rebuild / re-check the
result.
It seems that when using fixed-output derivations, only the name
attribute matters. I think there are use cases (like the example
above) that requires other derivation inputs to be considered
important enough to do a rebuild.
Vladimir, does it make sense to you now?
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 09/20/2014 09:55 PM, Bjørn Forsman wrote: Since fetchurl (via the nixos tarball mirror) downloads the same old file (even though it creates a new store path), the result outputHash is the same as before. So evaluation doesn't stop but continues on with old/wrong data. If fetchurl *did* download any other data, then it would fail while checking the hash. That was my point. Nixos tarball mirror may be different and download the same content, but in any case that doesn't change the plain fact of the first sentence of this paragraph. I think we all agree on how fetchurl works, but I still have no idea what you desire. I may be an exception, but I don't see what could be done between the current fixed-output and standard derivations. Either you do know what exactly the output should look like and then it doesn't really matter how you arrive at it, or you don't know it and then you have a standard derivation. Vladimir smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 21 September 2014 08:15, Vladimír Čunát vcu...@gmail.com wrote:
On 09/20/2014 09:55 PM, Bjørn Forsman wrote:
Since fetchurl (via the nixos tarball mirror) downloads the same old
file (even though it creates a new store path), the result outputHash
is the same as before. So evaluation doesn't stop but continues on
with old/wrong data.
If fetchurl *did* download any other data, then it would fail while checking
the hash. That was my point. Nixos tarball mirror may be different and
download the same content, but in any case that doesn't change the plain
fact of the first sentence of this paragraph.
I think we all agree on how fetchurl works, but I still have no idea what
you desire. I may be an exception, but I don't see what could be done
between the current fixed-output and standard derivations. Either you do
know what exactly the output should look like and then it doesn't really
matter how you arrive at it, or you don't know it and then you have a
standard derivation.
I'll try to explain the issue with an example. Given the following expression:
with (import nixpkgs { });
stdenv.mkDerivation rec {
name = foo-0.0;
outputHashAlgo = sha256;
outputHash = 0sldaa6dd6dsfm1c7f8z9i32dfhn53aw1kyrlid1naa7hb6biy4v;
outputHashMode = recursive;
buildCommand = ''
mkdir -p $out
echo hello $out/file
'';
}
Build it. Then change the echo hello $out/file to echo foobar
$out/file. The output will clearly change, but nix has cached the
result and will not do a rebuild. That's bad!
(You may at this point wonder, why use fixed-output derivation at all?
It's because the buildCommand, as is the case when packaging grails
apps, will download dependencies from the internet, so a standard
derivation will fail.)
I can work around this caching this by adding some hackery:
stdenv.mkDerivation rec {
extraHash = builtins.hashString sha256 (buildCommand);
name = foo-${extraHash}-0.0;
This results in a derivation that gets rebuilt/checked whenever
buildCommand changes. This is what I want.
But this solution feels very hackish. So my question is whether it's
possible to tell nix that certain derivation inputs affect the output,
so that when the inputs change, nix has to rebuild / re-check the
result.
It seems that when using fixed-output derivations, only the name
attribute matters. I think there are use cases (like the example
above) that requires other derivation inputs to be considered
important enough to do a rebuild.
Vladimir, does it make sense to you now?
Best regards,
Bjørn Forsman
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
Hi, How difficult is it to implement this? While packaging an application that downloads a bunch of dependencies during the build (like all grails apps do), I realized that I needed this. What I did first was to write a fixed-output derivation to let the application download its deps. Then I locked it down by fixing the hash. But now the problem is that even when the application changes its dependency specification, nix will not attempt to download the new deps, because it *only* looks at the output hash. I really need it to consider some of its inputs too, because the dependency spec comes from the input (application source). IMHO, something like this could also be useful for fetchurl: Let's say we made it also depend on the *basename* of url, then it'd re-download on package bumps but *not* on mirror updates. (The fact that fetchurl ignores package bumps is slightly confusing.) Best regards, Bjørn Forsman ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
Hi. On 09/20/2014 08:34 PM, Bjørn Forsman wrote: IMHO, something like this could also be useful for fetchurl: Let's say we made it also depend on the*basename* of url, then it'd re-download on package bumps but*not* on mirror updates. (The fact that fetchurl ignores package bumps is slightly confusing.) I don't have a clue what you want. If you need to download a different file, then the hash *will* change, and you have to rewrite it in the expression anyway. Vladimir smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 20 September 2014 20:50, Vladimír Čunát vcu...@gmail.com wrote: Hi. On 09/20/2014 08:34 PM, Bjørn Forsman wrote: IMHO, something like this could also be useful for fetchurl: Let's say we made it also depend on the*basename* of url, then it'd re-download on package bumps but*not* on mirror updates. (The fact that fetchurl ignores package bumps is slightly confusing.) I don't have a clue what you want. If you need to download a different file, then the hash *will* change, and you have to rewrite it in the expression anyway. No, fetchurl wont re-download if you change url. You have to update the hash to force a re-download. That's by design, because changing the mirror shouldn't cause a mass rebuild. But my realization was that we could have made it depend on the *basename* of url and still prevent a changing mirror to be mass-rebuild. For fetchurl this behaviour (ignoring inputs) is just a minor annoyance, but for packaging a grails app it completely breaks down. Best regards, Bjørn Forsman ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
fetchurl does depend on the base name, it's the nixos tarball mirror that doesn't div Original message /divdivFrom: Bjørn Forsman bjorn.fors...@gmail.com /divdivDate:09/20/2014 12:11 PM (GMT-08:00) /divdivTo: Vladimír Čunát vcu...@gmail.com /divdivCc: nix-dev nix-dev@lists.science.uu.nl /divdivSubject: Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs? /divdiv /divOn 20 September 2014 20:50, Vladimír Čunát vcu...@gmail.com wrote: Hi. On 09/20/2014 08:34 PM, Bjørn Forsman wrote: IMHO, something like this could also be useful for fetchurl: Let's say we made it also depend on the*basename* of url, then it'd re-download on package bumps but*not* on mirror updates. (The fact that fetchurl ignores package bumps is slightly confusing.) I don't have a clue what you want. If you need to download a different file, then the hash *will* change, and you have to rewrite it in the expression anyway. No, fetchurl wont re-download if you change url. You have to update the hash to force a re-download. That's by design, because changing the mirror shouldn't cause a mass rebuild. But my realization was that we could have made it depend on the *basename* of url and still prevent a changing mirror to be mass-rebuild. For fetchurl this behaviour (ignoring inputs) is just a minor annoyance, but for packaging a grails app it completely breaks down. Best regards, Bjørn Forsman ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 09/20/2014 09:11 PM, Bjørn Forsman wrote: No, fetchurl wont re-download if you change url. You have to update the hash to force a re-download. That's by design, because changing the mirror shouldn't cause a mass rebuild. But my realization was that we could have made it depend on the*basename* of url and still prevent a changing mirror to be mass-rebuild. For fetchurl this behaviour (ignoring inputs) is just a minor annoyance, but for packaging a grails app it completely breaks down. When I change the basename (and leave the hash the same), fetchurl *will* re-download, because the output name of the file does change. (It will fail, so I see little point anyway.) Vladimir smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 20 September 2014 21:21, Shea Levy s...@shealevy.com wrote: fetchurl does depend on the base name, it's the nixos tarball mirror that doesn't Ah, I didn't know that. I did a quick test now, which verified your statement. When changing the url, but keeping the hash, fetchurl re-downloads. A new store path is created with *new name*, but *old file contents*. On 20 September 2014 21:26, Vladimír Čunát vcu...@gmail.com wrote: [...] When I change the basename (and leave the hash the same), fetchurl *will* re-download, because the output name of the file does change. (It will fail, so I see little point anyway.) Actually, it doesn't fail. (At least in my test.) ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fixed-output derivation that *also* depend on (some of) its inputs?
On 20 September 2014 21:45, Bjørn Forsman bjorn.fors...@gmail.com wrote: [...] On 20 September 2014 21:26, Vladimír Čunát vcu...@gmail.com wrote: [...] When I change the basename (and leave the hash the same), fetchurl *will* re-download, because the output name of the file does change. (It will fail, so I see little point anyway.) Actually, it doesn't fail. (At least in my test.) Since fetchurl (via the nixos tarball mirror) downloads the same old file (even though it creates a new store path), the result outputHash is the same as before. So evaluation doesn't stop but continues on with old/wrong data. My idea was that if the basename of url was a part of the outputHash, fetchurl would actually fail in this scenario. This would prevent version bumps that have no effect, because of forgetting to update the hash. And like I said in the first post, for packaging a grails app, also taking the input into account when checking the output is a must. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
