Re: [Nix-dev] Auto-generated expressions for applications

[lewo]

Freddy Rietdijk  writes:

> Hi,
>
> At several places in Nixpkgs we use auto-generated data, mostly for the
> larger package sets like Haskell. Sometimes we also use auto-generated sets
> for applications that may need different versions than are offered in the
> main package sets. In the past months several issues/PR's have been opened
> to add rather large Python applications to Nixpkgs, generated using
> `pypi2nix`.
>
> Using such tool to generate the expressions has some very big advantages.
> However, there's also a disadvantage. The expressions shouldn't only be
> updated when there's a new release of the application, but also whenever
> there's security updates in any of its generated dependencies, which the
> application maintainer now has to keep track of. Therefore, I find it quite
> a risk to have separate package sets. At the same time, we'll also likely
> run behind in the main package sets every once in a while.

Regarding the security updates, if we are able to know which versions of
libraries are used in the `pythonPackage` set and in all 'application
pythonPackage sets', in case of a CVE on a particular package version,
we could know which applications are impacted. We then have to fix them
or mark them as broken.
I don't really know how it is hard to get all python module versions
used by applications and if this would be feasable.

> What do you think of this issue? Any suggestions how we can improve this?
> Maybe we could have a server/bot that runs update scripts and opens a PR
> whenever there's an actual diff?
>
> Freddy
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> https://mailman.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Auto-generated expressions for applications

[lewo]

Freddy Rietdijk  writes:

> Hi,
>
> At several places in Nixpkgs we use auto-generated data, mostly for the
> larger package sets like Haskell. Sometimes we also use auto-generated sets
> for applications that may need different versions than are offered in the
> main package sets. In the past months several issues/PR's have been opened
> to add rather large Python applications to Nixpkgs, generated using
> `pypi2nix`.
>
> Using such tool to generate the expressions has some very big advantages.
> However, there's also a disadvantage. The expressions shouldn't only be
> updated when there's a new release of the application, but also whenever
> there's security updates in any of its generated dependencies, which the
> application maintainer now has to keep track of. Therefore, I find it quite
> a risk to have separate package sets. At the same time, we'll also likely
> run behind in the main package sets every once in a while.

As an example, `openstackclient` needs a version of the `requests`
library which is older than our `pythonPackages.requests`. We then need
to add another version but we don't want to mix different versions in
the python packages set (to avoid conflicts). In this context, it seems
the only way to bring `openstackclient` was to provide its dependencies
as 'private' dependencies.

I think several `golang` applications are also packaged in `nixpkgs` by
"vendorizing" their dependencies.
How dependencies of these applications are managed?

> What do you think of this issue? Any suggestions how we can improve this?
> Maybe we could have a server/bot that runs update scripts and opens a PR
> whenever there's an actual diff?

I don't know how but I think it would be nice to be able to know the
python module versions that are currently used by all applications.

> Freddy
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> https://mailman.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Shutting down prs.nix.gsc.io

[lewo]

I wonder how many resources you have used.

600GB of storage and 3TB of upload for 2weeks. Is it right?
And how many vcpus, and rams did you use/need?

Antoine.

Graham Christensen  writes:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
>
> Hey everyone, :)
>
> I am touched by your offers to contribute to help pay for it. I am so
> thrilled to be part of a community that takes care of its own. It
> reminds me of this talk: https://www.youtube.com/watch?v=Xe1TZaElTAs.
>
> However, please instead take your generosity and perhaps donate to the
> NixOS Foundation: https://nixos.org/nixos/foundation.html or another
> worthy cause (up to you to know what that means.)
>
> Here is exactly how much the experiment cost:
>
> Usage Type| Usage   |  Price| Total
> 
> S3 PUT Requests   | 2,066,983   | $0.005/1,000  | $ 10.33
> S3 GET Requests   | 4,812,149   | $0.004/10,000 | $  1.92
> S3 Timed Storage  |   603 GB-Mo | $0.023/GB-mo  | $ 13.87
> Data Transfer In  | 1,907.028 GB| $0/GB | $  0.00
> Data Transfer Out | 3,698.464 GB| $0.090/GB | $332.86
> - -
>   $358.98
>
> Thank you so much,
> Graham Christensen
> -BEGIN PGP SIGNATURE-
>
> iQIcBAEBCAAGBQJYkdbAAAoJEAYSHTZv6UNcTcgP/38I07FjuexTkTIaRnzMawsN
> My6+IfWWzv+MToaiYLLaj5gMXplrHIuzdUVxq0LXAxqLDQdMbw3txuUai4gglFGS
> gWss7/3yYEyws0viT0OmWixq3+bDs57bAyC2k6LPt4QzxgFiLeAk1fyk7e2t1n6S
> 3GOj7HW000kjK7caxqTsKMORrk1WsQ6i1R+XupwJw8NU+0mFvVM/C6B3d5w8lP2S
> B9x0EcSV63rUh4IAio7EqU99KoJcLmc/HaE9uq4zYTFMhzcV1imV4bj9s+Qq6MQf
> 7amIu5br7ntBwTWDzdGmiMLbIbSI5IB4hzfDOedkavpWfwfj7EBR9j4cK9PHbo0+
> PoGR2x10wma6Sa0DrDgmRZx8WJypMXGYbgGZQCxr04wlYMh/hl7z9IhPZ8Z37GXx
> ote01hRdS82hfWgRIagkBJG9L5MnVqS5WNLHmo4tg2sf7ADPJoXUz7MO/zaGx2Dh
> dKzPDAA+cLOwcWI+rIoQTOIWTlskyjbJpymoNbhm1MZhqsGK4i+MLwFrDla8k6nh
> uhWg8lCAwWEJ3QqGRdeW3Pp2+d9KAIAN5QLFRFmW8s8fmLxLdbo2bPr/sw7NSTdk
> p7ZMJhC6K8bJkCleSxxGSsDvX5icokL0ZaaXVlZcyeZo8/ODQCBfxbC6M57uSrig
> qhsZ2InUPjQGTJZeozOb
> =Iu/C
> -END PGP SIGNATURE-
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] Openstack modules review

[lewo]

Hello,

I've continued works on Openstack modules started by domenkozar. We are
now able to easily deploy a basic Openstack all-in-one node.

There is a test that deploys Nova, Glance, Keystone and Neutron with the
linuxbridge driver. It then boots a VM and pings it to check its
connectivity.

These modules are not production ready yet but they can be used to
provide a dev environment, such as a devstack (without all problems that
you could imagine when Python packages are deployed via shell
scripts!).

So, I'm looking for people that would like to review this PR [1],
indicate which points have to be improved.

Thanks

[1] https://github.com/NixOS/nixpkgs/pull/19529
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Build dependent haskell packages

[lewo]

hi Peter,

> what version of NixOS do you use? The "ghc7.8.3-0.12-shared" bit of the
> package name suggests it's something rather old.

Exactly, it's something rather old since I'm using 14.12.

>  >   {
>  > packageOverrides = pkgs: rec {
>  >   haskellPackages = with pkgs.haskellPackages; pkgs.haskellPackages // 
> rec {
>  >   xmonad = callPackage /home/lewo/repos/xmonad {};
>  >   xmonadContrib = callPackage /home/lewo/repos/XMonadContrib {};
>  >   };
>  > };
>  >   }
>
> That override won't work (evidently). It replaces the value of xmonad
> that you see during "nix-env", but it won't replace the value of xmonad
> that xmonadContrib sees while binding its dependencies. The Nixpkgs
> manual covers these details in [1], but it refers to release-15.09 or
> later of Nixpkgs, I'm afraid.

I finally overrided the xmonad dependency in the xmonadContrib
callPackage function as following:
xmonadContrib = callPackage /home/lewo/repos/XMonadContrib { xmonad = xmonad };

Thanks for mentionning the dependencies binding problem at build time
since this has allowed me to solve my difficulty.

lewo.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] Build dependent haskell packages

[lewo]

Hello,

I'm trying to install xmonad-0.12 and xmonad-contrib-0.12 by building a
nix package for each of these ones. To build nix packages, I use cabal2nix.

I first build and install xmonad-0.12 nix package as illustrated:
  nix-env -qas -A nixos.pkgs.haskellPackages.xmonad
  IP-  haskell-xmonad-ghc7.8.3-0.12-shared

I then trying to install xmonad-contrib-0.12 but I've a dependency
problem:
  nix-env -i -A nixos.pkgs.haskellPackages.xmonadContrib
  installing ‘haskell-xmonad-contrib-ghc7.8.3-0.12-shared’
  these derivations will be built:

/nix/store/nc6c7s900y4sfr5ahyhzlak52vdklv3y-haskell-xmonad-contrib-ghc7.8.3-0.12-shared.drv
  building path(s) 
‘/nix/store/974kiz5pv8xrnlxrh82zc9jx6srb0abz-haskell-xmonad-contrib-ghc7.8.3-0.12-shared’
  building 
/nix/store/974kiz5pv8xrnlxrh82zc9jx6srb0abz-haskell-xmonad-contrib-ghc7.8.3-0.12-shared
  unpacking sources
  unpacking source archive 
/nix/store/m99qm4zwsyda3c8gqa4hj5nk03fv9z7i-XMonadContrib
  source root is XMonadContrib
  patching sources
  configuring
  [1 of 1] Compiling Main ( Setup.lhs, 
/tmp/nix-build-haskell-xmonad-contrib-ghc7.8.3-0.12-shared.drv-0/Main.o )
  Linking Setup ...
  configure flags: --enable-split-objs --disable-library-profiling 
--enable-shared --enable-library-vanilla --enable-executable-dynamic 
--enable-tests  
--ghc-option=-optl=-Wl,-rpath=/nix/store/974kiz5pv8xrnlxrh82zc9jx6srb0abz-haskell-xmonad-contrib-ghc7.8.3-0.12-shared/lib/ghc-7.8.3/xmonad-contrib-0.12
  Configuring xmonad-contrib-0.12...
  Setup: At least the following dependencies are missing:
  xmonad ==0.12.*
  builder for 
‘/nix/store/nc6c7s900y4sfr5ahyhzlak52vdklv3y-haskell-xmonad-contrib-ghc7.8.3-0.12-shared.drv’
 failed with exit code 1
  error: build of 
‘/nix/store/nc6c7s900y4sfr5ahyhzlak52vdklv3y-haskell-xmonad-contrib-ghc7.8.3-0.12-shared.drv’
 failed

  In order to override upstream version, I use the following config
  file:
  cat ~/.nixpkgs/config.nix 
  {
packageOverrides = pkgs: rec {
  haskellPackages = with pkgs.haskellPackages; pkgs.haskellPackages // rec {
  xmonad = callPackage /home/lewo/repos/xmonad {};
  xmonadContrib = callPackage /home/lewo/repos/XMonadContrib {};
  };
};
  }

I don't understand why this dependency is not managed by nix since it
is specified in the cabal file:
  grep "xmonad>" xmonad-contrib.cabal 
  xmonad>=0.12   && < 0.13,

Any ideas about this?
Should I use another way to build them?

Thanks in advance,
lewo.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev