Re: [Nix-dev] importing nixpkgs from derivation dumps large path

2017-03-03 Thread Ganesh Sittampalam 
On 02/03/2017 10:53, Eelco Dolstra wrote:

> On 03/02/2017 08:01 AM, Ganesh Sittampalam wrote:
> 
>> I guess that in this case the large tree is the import of the nixpkgs
>> tree. Is there any work around or better way to achieve what I want? 
> 
> Nix 1.12 has a builtin
> fetchgit function that allows you to say
> 
>   with import (builtins.fetchgit {
> url = ;
> rev = "...";
>   }) {};
>   ...
> 
> (I just pushed a fix to allow "url" to refer to paths.)

Thanks! I've had a play with this, but it reruns the git fetch every
time, which takes a few seconds, nearly as long as the nixpkgs fetchgit
- I guess because no hash is specified?

I can see that this behaviour would also be useful in some cases as I
could just update the git ref live, but the evaluation time cost hurts
with what I'm currently trying to do.

Cheers,

Ganesh


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] importing nixpkgs from derivation dumps large path

2017-03-02 Thread Eelco Dolstra 
Hi,

On 03/02/2017 08:01 AM, Ganesh Sittampalam wrote:

> I guess that in this case the large tree is the import of the nixpkgs
> tree. Is there any work around or better way to achieve what I want? 

The better way is to use builtins.fetchTarball:

  with import (fetchTarball {
url = https://github.com/NixOS/nixpkgs/archive/264d42b.tar.gz;
sha256 = "19i77afcns8mf88vkxvm3jvkzdkf5x8p8kxnnivhd9l4kslkq3v5";
  }) {};

  {
env =
  stdenv.mkDerivation rec {
...
  };
};
  }

This also prevents a double evaluation of Nixpkgs. However, this won't work if
you really need to fetch from a local Git repository. Nix 1.12 has a builtin
fetchgit function that allows you to say

  with import (builtins.fetchgit {
url = ;
rev = "...";
  }) {};
  ...

(I just pushed a fix to allow "url" to refer to paths.)

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] importing nixpkgs from derivation dumps large path

2017-03-02 Thread Danylo Hlynskyi 
as a workaround you can use fetchFromGitHub or

fetchTarball "https://github.com/NixOS/nixpkgs/archive/${commit}.tar.gz;



2017-03-02 9:01 GMT+02:00 Ganesh Sittampalam :

> Hi,
>
> I'm trying to maintain a set of development environments using
> nix-shell. For each environment, I want to keep them pinned at a
> particular git revision of nixpkgs until I explicitly upgrade that
> environment, so I can check that everything still works properly.
>
> I used to manage this manually with a comment on myEnvFun environments.
> While migrating them to nix-shell, I learnt that I can use "import from
> derivation" for this instead and have it expressed within the nix language:
>
>  {
>env =
>  let pkgs =
>   import
>  (with import  {} ;
>   fetchgit {
> url = ;
> rev = "264d42b";
> sha256 =
>  "19i77afcns8mf88vkxvm3jvkzdkf5x8p8kxnnivhd9l4kslkq3v5";
>   }) {};
>
>  in with pkgs ; stdenv.mkDerivation rec {
>  name = "haskell-http-ghc710";
>  buildInputs = [ stdenv pkgconfig gcc binutils
>  haskell.compiler.ghc7102
>  haskellPackages.cabal-install
>  zlib ];
>  };
>  }
>
> When I instantiate this, e.g. with
>
>  nix-instantiate -I nixpkgs= demo.nix -A env
>
> it takes about 5-10s and prints out this message, though it does complete:
>
> warning: dumping very large path (> 256 MiB); this may run out of memory
>
> I'm aware that this error is typically associated with trying to import
> large files or large trees into the store:
>
> https://github.com/NixOS/nix/issues/358
> https://github.com/NixOS/nixpkgs/issues/12243
>
> I guess that in this case the large tree is the import of the nixpkgs
> tree. Is there any work around or better way to achieve what I want? The
> time taken is annoying particularly because I have a lot of these shells
> that I sometimes want to rebuild all together.
>
> Cheers,
>
> Ganesh
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] importing nixpkgs from derivation dumps large path

2017-03-01 Thread Ganesh Sittampalam 
Hi,

I'm trying to maintain a set of development environments using
nix-shell. For each environment, I want to keep them pinned at a
particular git revision of nixpkgs until I explicitly upgrade that
environment, so I can check that everything still works properly.

I used to manage this manually with a comment on myEnvFun environments.
While migrating them to nix-shell, I learnt that I can use "import from
derivation" for this instead and have it expressed within the nix language:

 {
   env =
 let pkgs =
  import
 (with import  {} ;
  fetchgit {
url = ;
rev = "264d42b";
sha256 =
 "19i77afcns8mf88vkxvm3jvkzdkf5x8p8kxnnivhd9l4kslkq3v5";
  }) {};

 in with pkgs ; stdenv.mkDerivation rec {
 name = "haskell-http-ghc710";
 buildInputs = [ stdenv pkgconfig gcc binutils
 haskell.compiler.ghc7102
 haskellPackages.cabal-install
 zlib ];
 };
 }

When I instantiate this, e.g. with

 nix-instantiate -I nixpkgs= demo.nix -A env

it takes about 5-10s and prints out this message, though it does complete:

warning: dumping very large path (> 256 MiB); this may run out of memory

I'm aware that this error is typically associated with trying to import
large files or large trees into the store:

https://github.com/NixOS/nix/issues/358
https://github.com/NixOS/nixpkgs/issues/12243

I guess that in this case the large tree is the import of the nixpkgs
tree. Is there any work around or better way to achieve what I want? The
time taken is annoying particularly because I have a lot of these shells
that I sometimes want to rebuild all together.

Cheers,

Ganesh
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev