Re: [Nix-dev] Request for comments: pinky-promise determinism
For dirty dirty hacks, you can set __noChroot = true and get access to the network. Okay, now I know how to implement it. I'll try to keep it clean. Thanks -- Georges ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Request for comments: pinky-promise determinism
I don't like to use the git hash, as it is SHA1 and not fit for use wrt integrity. I think that would be a regression wrt the complex tricks that Georges referred to. Would it be possible to create a git wrapper that does the make_deterministic_repo step in a transparent manner for cargo and have this git be in the build environment? Alexander On Fri, Jan 2, 2015 at 9:07 PM, Wout Mertens wout.mert...@gmail.com wrote: Another use-case: providing the same input hash, based only on version, for gcc and cross-gcc on another platform. Ditto for ccache and distcc. On Fri, Jan 2, 2015, 14:56 Shea Levy s...@shealevy.com wrote: For dirty dirty hacks, you can set __noChroot = true and get access to the network. On Jan 2, 2015, at 1:09 PM, Georges Dubus georges.du...@gmail.com wrote: Hello everyone I would like to propose compromise in the purity rules of non-fixed-output derivations, and hear what you think about it. # Rationale There are a few situations where derivations play the role of fixed-output derivation, but the hash of their output is not fixed. Some examples: - fetchgit derivations when the .git must be kept. The .git directory is incredibly hard to make deterministic, as this require tweaking with implementation details: purging any commit that might have been downloaded from the server, that may have no link with the reference we are using. - cargo, the package manager for the rust language, uses git to download its database, and to check it is up-to-date. The same problem as with fetchgit arise, with the increased trouble that we are now tweaking the implementation detail of an implementation detail. However, we can trust that, even though the .git is not binary identical in each situation, the result of the git commands we could use in the packaging task is always the same. # Proposition I propose a new kind of derivation that would be identical to the current non-fixed-output derivation, but without any restriction on its access to the outside world. The documentation should state that this kind of derivation is dangerous, and should only be used when a trustworthy tool is used (since the tool is trusted to be deterministic in its behaviour). This new derivation could be used for dirty hacks, but this should be discouraged by the documentation, and never accepted inside nixpkgs. # Conclusion The inclusion of this new kind of derivation would allow a satisfying implementation of leaveDotGit for fetchgit, one that does not rely on complex tricks[1], and allow me to implement cargo support without relying on non-future-proof internals tweaking. However, this would be at the cost of including a new kind of derivation that is much less satisfying, and that could, if misused, come back to bite us. I'd love to hear what you think about it. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/nix-prefetch-git#L198 -- Georges Dubus ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Request for comments: pinky-promise determinism
For dirty dirty hacks, you can set __noChroot = true and get access to the network. On Jan 2, 2015, at 1:09 PM, Georges Dubus georges.du...@gmail.com wrote: Hello everyone I would like to propose compromise in the purity rules of non-fixed-output derivations, and hear what you think about it. # Rationale There are a few situations where derivations play the role of fixed-output derivation, but the hash of their output is not fixed. Some examples: - fetchgit derivations when the .git must be kept. The .git directory is incredibly hard to make deterministic, as this require tweaking with implementation details: purging any commit that might have been downloaded from the server, that may have no link with the reference we are using. - cargo, the package manager for the rust language, uses git to download its database, and to check it is up-to-date. The same problem as with fetchgit arise, with the increased trouble that we are now tweaking the implementation detail of an implementation detail. However, we can trust that, even though the .git is not binary identical in each situation, the result of the git commands we could use in the packaging task is always the same. # Proposition I propose a new kind of derivation that would be identical to the current non-fixed-output derivation, but without any restriction on its access to the outside world. The documentation should state that this kind of derivation is dangerous, and should only be used when a trustworthy tool is used (since the tool is trusted to be deterministic in its behaviour). This new derivation could be used for dirty hacks, but this should be discouraged by the documentation, and never accepted inside nixpkgs. # Conclusion The inclusion of this new kind of derivation would allow a satisfying implementation of leaveDotGit for fetchgit, one that does not rely on complex tricks[1], and allow me to implement cargo support without relying on non-future-proof internals tweaking. However, this would be at the cost of including a new kind of derivation that is much less satisfying, and that could, if misused, come back to bite us. I'd love to hear what you think about it. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/nix-prefetch-git#L198 https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/nix-prefetch-git#L198 -- Georges Dubus ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Request for comments: pinky-promise determinism
Hello everyone I would like to propose compromise in the purity rules of non-fixed-output derivations, and hear what you think about it. # Rationale There are a few situations where derivations play the role of fixed-output derivation, but the hash of their output is not fixed. Some examples: - fetchgit derivations when the .git must be kept. The .git directory is incredibly hard to make deterministic, as this require tweaking with implementation details: purging any commit that might have been downloaded from the server, that may have no link with the reference we are using. - cargo, the package manager for the rust language, uses git to download its database, and to check it is up-to-date. The same problem as with fetchgit arise, with the increased trouble that we are now tweaking the implementation detail of an implementation detail. However, we can trust that, even though the .git is not binary identical in each situation, the result of the git commands we could use in the packaging task is always the same. # Proposition I propose a new kind of derivation that would be identical to the current non-fixed-output derivation, but without any restriction on its access to the outside world. The documentation should state that this kind of derivation is dangerous, and should only be used when a trustworthy tool is used (since the tool is trusted to be deterministic in its behaviour). This new derivation could be used for dirty hacks, but this should be discouraged by the documentation, and never accepted inside nixpkgs. # Conclusion The inclusion of this new kind of derivation would allow a satisfying implementation of leaveDotGit for fetchgit, one that does not rely on complex tricks[1], and allow me to implement cargo support without relying on non-future-proof internals tweaking. However, this would be at the cost of including a new kind of derivation that is much less satisfying, and that could, if misused, come back to bite us. I'd love to hear what you think about it. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/nix-prefetch-git#L198 -- Georges Dubus ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Request for comments: pinky-promise determinism
I've talked to git (irc) long time ago, they are pretty sure that the git
hash would serve well as alternative to a sha256 or md5 hash.
Thus adding an implementation for
mkDerivation {
git_hash = ; # instead of fixed output hash
}
and have the nix implementation check for $out/.git and, git status
returning nothing and HEAD being the desired hash would be much easier
to use.
Most Vim plugins also are stored on github.
Marc Weber
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Request for comments: pinky-promise determinism
Another use-case: providing the same input hash, based only on version, for gcc and cross-gcc on another platform. Ditto for ccache and distcc. On Fri, Jan 2, 2015, 14:56 Shea Levy s...@shealevy.com wrote: For dirty dirty hacks, you can set __noChroot = true and get access to the network. On Jan 2, 2015, at 1:09 PM, Georges Dubus georges.du...@gmail.com wrote: Hello everyone I would like to propose compromise in the purity rules of non-fixed-output derivations, and hear what you think about it. # Rationale There are a few situations where derivations play the role of fixed-output derivation, but the hash of their output is not fixed. Some examples: - fetchgit derivations when the .git must be kept. The .git directory is incredibly hard to make deterministic, as this require tweaking with implementation details: purging any commit that might have been downloaded from the server, that may have no link with the reference we are using. - cargo, the package manager for the rust language, uses git to download its database, and to check it is up-to-date. The same problem as with fetchgit arise, with the increased trouble that we are now tweaking the implementation detail of an implementation detail. However, we can trust that, even though the .git is not binary identical in each situation, the result of the git commands we could use in the packaging task is always the same. # Proposition I propose a new kind of derivation that would be identical to the current non-fixed-output derivation, but without any restriction on its access to the outside world. The documentation should state that this kind of derivation is dangerous, and should only be used when a trustworthy tool is used (since the tool is trusted to be deterministic in its behaviour). This new derivation could be used for dirty hacks, but this should be discouraged by the documentation, and never accepted inside nixpkgs. # Conclusion The inclusion of this new kind of derivation would allow a satisfying implementation of leaveDotGit for fetchgit, one that does not rely on complex tricks[1], and allow me to implement cargo support without relying on non-future-proof internals tweaking. However, this would be at the cost of including a new kind of derivation that is much less satisfying, and that could, if misused, come back to bite us. I'd love to hear what you think about it. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/nix-prefetch-git#L198 -- Georges Dubus ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
