Re: [Nix-dev] git in nix is broken...

[Dmitry Kalinkin via nix-dev]

Same can be achieved with makeWrapper
(http://nixos.org/nixpkgs/manual/#ssec-stdenv-functions)
Also, there is propagatedBuildInputs, but it is limited to Nix build
environment.

On Mon, Jul 10, 2017 at 2:36 PM, Alexander Schmolck via nix-dev
 wrote:
> Yes, git filter-branch failing because e.g. sed is either not
> installed at all or not in PATH is what I meant.
>
> I wasn't aware that this is "intended behavior", I thought that nix
> packages were intended to be self-contained other than explicitly
> declared dependencies
>
> The way I ran into this was by building a docker image for a python
> application that has a runtime dependency on git with
> dockerTools.buildImage (code is here:
> https://github.com/smarkets/marge/blob/master/dockerize.nix). Most git
> commands work fine, but not ones that use the few commands that are
> implemented as shell scripts. Pulling in the branch in the PR via
> fetchTarball fixes the problem, but what would be the preferred way of
> doing this?
>
> alexander
>
> On 10 July 2017 at 18:47, Eelco Dolstra via nix-dev
>  wrote:
>> Hi,
>>
>> On 07/10/2017 07:25 PM, Alexander Schmolck via nix-dev wrote:
>>
>>> ... because it appears to be missing dependencies like sed for
>>> commands such as git-filter-branch which are implemented as bash
>>> scripts.
>>
>> Broken how? What doesn't work?
>>
>> I guess you mean that some things fail if you execute with a $PATH that 
>> doesn't
>> contain the basic Unix utilities. This is likely the case for most shell 
>> code in
>> Nixpkgs.
>>
>> --
>> Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
>> ___
>> nix-dev mailing list
>> nix-dev@lists.science.uu.nl
>> https://mailman.science.uu.nl/mailman/listinfo/nix-dev
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> https://mailman.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Nix-commits] [NixOS/nixpkgs] 8c6e91: citrix-receiver: fix wrapper, direct dl url, $PWD ...

[Dmitry Kalinkin]

  Branch: refs/heads/release-17.03
  Home:   https://github.com/NixOS/nixpkgs
  Commit: 8c6e9177a468618a63836f1e28576e682f93f216
  
https://github.com/NixOS/nixpkgs/commit/8c6e9177a468618a63836f1e28576e682f93f216
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2017-03-26 (Sun, 26 Mar 2017)

  Changed paths:
M pkgs/applications/networking/remote/citrix-receiver/default.nix

  Log Message:
  ---
  citrix-receiver: fix wrapper, direct dl url, $PWD needs escaping

(edited by @obadz, closes #24320)

(cherry picked from commit 7035325c6243f5630fb8ca823f861057b2ee40d1)


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

[Nix-commits] [NixOS/nixpkgs] 703532: citrix-receiver: fix wrapper, direct dl url, $PWD ...

[Dmitry Kalinkin]

  Branch: refs/heads/master
  Home:   https://github.com/NixOS/nixpkgs
  Commit: 7035325c6243f5630fb8ca823f861057b2ee40d1
  
https://github.com/NixOS/nixpkgs/commit/7035325c6243f5630fb8ca823f861057b2ee40d1
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2017-03-26 (Sun, 26 Mar 2017)

  Changed paths:
M pkgs/applications/networking/remote/citrix-receiver/default.nix

  Log Message:
  ---
  citrix-receiver: fix wrapper, direct dl url, $PWD needs escaping

(edited by @obadz, closes #24320)


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

Re: [Nix-dev] How do you work on big packages?

[Dmitry Kalinkin]

On Tue, Mar 21, 2017 at 10:01 AM, Kosyrev Serge <skosy...@ptsecurity.com> wrote:
>
> Dmitry Kalinkin writes:
>>  On 17 Mar 2017, at 17:44, Vladimír Čunát <vcu...@gmail.com> wrote:
>>
>>  I do believe the intention was for "SW distribution" etc, at least
>>  primarily, and the suitability for development is a by-product due to
>>  some properties, e.g. easy (non-)mixing of development and stable
>>  versions/configs. Marc can surely remember the earlier days of NixOS.
>>
>> So nix has “nix copy” functionality to do the distribution part. But the 
>> rest of the nix system allows to describe a
>> relatively general computation process in Unix-like environment. I use nix 
>> derivations to run a numerical
>> calculation code and store intermediate steps in outputs. I imagine, there 
>> are some “SW distribution”-oriented
>> users facing more resistance when using nix than me doing my thing. I also 
>> think that the nix/NixOS community
>> will have lots of people who appreciate generality of nix as a tool.
>>
>>  It's even possible to use nix-build instead of make to compile
>>  individual files, but there it just doesn't seem to be very suitable…
>>
>> Yes. Like https://github.com/edolstra/nix-make . I wonder what didn’t work 
>> out.
>
> Garbage management becomes complicated from a human perspective, I guess.
>
> Perhaps, Nix could adopt a generational GC approach, to avoid mixing
> long-lived packages-related-files and short-lived build-related files..
>
Yes. I think a good way to address this would be to have nix support
external stores. That way nix-make can store its artefacts separately.
Also this would allow to store data files on a separate volume
optimised for big files.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-commits] [NixOS/nix] 5bf968: override rx directory permissions in deletePath()

[Dmitry Kalinkin]

  Branch: refs/heads/1.11-maintenance
  Home:   https://github.com/NixOS/nix
  Commit: 5bf9689e0c9bd294af107039fc88ddb003e7328e
  
https://github.com/NixOS/nix/commit/5bf9689e0c9bd294af107039fc88ddb003e7328e
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-12-19 (Mon, 19 Dec 2016)

  Changed paths:
M src/libutil/util.cc

  Log Message:
  ---
  override rx directory permissions in deletePath()

This fixes instantiation of pythonPackages.pytest that produces a
directory with less permissions during one of it's tests that leads to
a nix error like:

error: opening directory 
‘/tmp/nix-build-python2.7-pytest-2.9.2.drv-0/pytest-of-user/pytest-0/testdir/test_cache_failure_warns0/.cache’:
 Permission denied
(cherry picked from commit f91748ba737c9d99b407572756cc003e4d956b8e)


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

Re: [Nix-dev] Python: getting rid of PYTHONPATH in Nixpkgs

[Dmitry Kalinkin]

> On 01 Nov 2016, at 11:56, Freddy Rietdijk <freddyrietd...@fridh.nl> wrote:
> 
> 
> > To solve 2) you could also make a wrapper that combines selected outputs of 
> > the multiple output package without python itself. I think, this is what 
> > texlive does. That way all the burden of wrapping will be limited only to 
> > the multiple output packages.
> 
> I'm not sure whether I understand what you mean. What kind of wrapper do you 
> mean? What does it wrap, and what does it set? The problem with the multiple 
> outputs is that when you would add each to PYTHONPATH, the extra outputs 
> won't be importable unless the 'main' output is first. I don't see how that 
> can be solved. One that sets PYTHONPATH? Or some kind of package
I had in mind a simple ```combine [blah.foo blah.bar]``` that would produce a 
derivation with symlinks to both $foo/lib/python2.7/blah/foo and 
$foo/lib/python2.7/blah/bar, so multiple outputs are combined back and can be 
importable with the current envHook setting the PYTHONPATH. This is similar to 
the current python wrapper that additionally throws in the python interpreter 
itself along with wrapper to set PYTHONHOME.
> 
> > As for the first point, I couldn’t very well understand what the problem 
> > is. Are you talking about conflict between instances of a package that were 
> > built against different version of the python? Then shouldn’t `--prefix` 
> > override the incompatible version, since, as you said, order matters?
> 
> Example with discussion: https://github.com/NixOS/nixpkgs/pull/17749
> 
> 
> On Tue, Nov 1, 2016 at 4:12 PM, Dmitry Kalinkin <dmitry.kalin...@gmail.com> 
> wrote:
> 
> > On 01 Nov 2016, at 06:22, Freddy Rietdijk <freddyrietd...@fridh.nl> wrote:
> >
> > Hi,
> >
> > Currently we use PYTHONPATH a lot in Nixpkgs to let applications and the 
> > interpreter find Python modules. This typically works fine, but there are 
> > problems with this method and so I would like to get rid of it and use only 
> > `python.withPackages` which uses `python.buildEnv`.
> >
> > The two main issues with the use of PYTHONPATH:
> >   • Before we used `--prefix $PYTHONPATH`, but this would leak 
> > PYTHONPATH into subprocesses, which is especially problematic when both 
> > parent and child depend on Python but of different versions.  `--set 
> > $PYTHONPATH` would solve that issue, but that makes it impossible to add 
> > other (impure) paths, which is an important feature. This also breaks 
> > alternative shells like `ipython`. This issue is currently solved by 
> > extending `sys.path` in the Python applications.
> >   • Limits the use of multiple outputs. When moving a module of a 
> > package into a separate output it becomes problematic to load this again, 
> > since just adding the module to PYTHONPATH typically doesn't work because 
> > the order matters. While Python modules are typically small, and build 
> > fast, rebuilding can take a lot of time in cases like `matplotlib` which 
> > supports multiple backends and is depended on by quite some packages.
> > A method that is more reliable is building an environment with symbolic 
> > links to all the modules, and wrapping the applications with a wrapper that 
> > sets PYTHONHOME. This is exactly what `python.buildEnv` does, and it solves 
> > both 1) and 2).
> >
> > `PYTHONPATH` is mainly constructed with the Python interpreter setupHook. 
> > It is used in `buildPythonPackage` for building the package, and after 
> > installing it is extended so the tests can run. Furthermore, `PYTHONPATH` 
> > is set by the `setupHook` when using `nix-shell` like `nix-shell -p 
> > pythonPackages.numpy`.
> >
> > I think we can get rid of the setupHook. For the building we can create an 
> > env. This would be able to support multiple outputs as inputs, but will be 
> > more expensive than setting PYTHONPATH. For the tests we do add the newly 
> > constructed package to PYTHONPATH; there's no way around it and it doesn't 
> > cause any problems either.
> >
> > The biggest impact will be on how `nix-shell` is used. It won't be possible 
> > anymore to use it as shown before, instead `nix-shell -p 
> > 'python.withPackages(ps:[ps.numpy])'` would have to be used. While it is 
> > possible to keep the setupHook (or use it as a shellHook) it will be 
> > unreliable in the case of multiple outputs.
> >
> > What do you think about this change? Do you see any problems with it? Any 
> > other suggestions?
> >
> > Freddy
> > ___
> > nix-dev mailing list
>

Re: [Nix-dev] Python: getting rid of PYTHONPATH in Nixpkgs

[Dmitry Kalinkin]

> On 01 Nov 2016, at 06:22, Freddy Rietdijk  wrote:
> 
> Hi,
> 
> Currently we use PYTHONPATH a lot in Nixpkgs to let applications and the 
> interpreter find Python modules. This typically works fine, but there are 
> problems with this method and so I would like to get rid of it and use only 
> `python.withPackages` which uses `python.buildEnv`.
> 
> The two main issues with the use of PYTHONPATH:
>   • Before we used `--prefix $PYTHONPATH`, but this would leak PYTHONPATH 
> into subprocesses, which is especially problematic when both parent and child 
> depend on Python but of different versions.  `--set $PYTHONPATH` would solve 
> that issue, but that makes it impossible to add other (impure) paths, which 
> is an important feature. This also breaks alternative shells like `ipython`. 
> This issue is currently solved by extending `sys.path` in the Python 
> applications.
>   • Limits the use of multiple outputs. When moving a module of a package 
> into a separate output it becomes problematic to load this again, since just 
> adding the module to PYTHONPATH typically doesn't work because the order 
> matters. While Python modules are typically small, and build fast, rebuilding 
> can take a lot of time in cases like `matplotlib` which supports multiple 
> backends and is depended on by quite some packages.
> A method that is more reliable is building an environment with symbolic links 
> to all the modules, and wrapping the applications with a wrapper that sets 
> PYTHONHOME. This is exactly what `python.buildEnv` does, and it solves both 
> 1) and 2).
> 
> `PYTHONPATH` is mainly constructed with the Python interpreter setupHook. It 
> is used in `buildPythonPackage` for building the package, and after 
> installing it is extended so the tests can run. Furthermore, `PYTHONPATH` is 
> set by the `setupHook` when using `nix-shell` like `nix-shell -p 
> pythonPackages.numpy`.
> 
> I think we can get rid of the setupHook. For the building we can create an 
> env. This would be able to support multiple outputs as inputs, but will be 
> more expensive than setting PYTHONPATH. For the tests we do add the newly 
> constructed package to PYTHONPATH; there's no way around it and it doesn't 
> cause any problems either.
> 
> The biggest impact will be on how `nix-shell` is used. It won't be possible 
> anymore to use it as shown before, instead `nix-shell -p 
> 'python.withPackages(ps:[ps.numpy])'` would have to be used. While it is 
> possible to keep the setupHook (or use it as a shellHook) it will be 
> unreliable in the case of multiple outputs.
> 
> What do you think about this change? Do you see any problems with it? Any 
> other suggestions?
> 
> Freddy
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev

Hi,

I’m far from being an expert in python, but here are my thoughts:

To solve 2) you could also make a wrapper that combines selected outputs of the 
multiple output package without python itself. I think, this is what texlive 
does. That way all the burden of wrapping will be limited only to the multiple 
output packages. As for the first point, I couldn’t very well understand what 
the problem is. Are you talking about conflict between instances of a package 
that were built against different version of the python? Then shouldn’t 
`--prefix` override the incompatible version, since, as you said, order matters?

Also, if you remove setupHook, won’t you also lose ability to specify python 
dependencies directly in buildInputs? Put that inside mkDerivation?

Dmitry
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-commits] [NixOS/nixpkgs] 5d78c8: numpy: enable numpy.distutils patch only if it's a...

[Dmitry Kalinkin]

  Branch: refs/heads/python-wip
  Home:   https://github.com/NixOS/nixpkgs
  Commit: 5d78c87a25fa430a2e39e7ba6d600f2edcf51c1f
  
https://github.com/NixOS/nixpkgs/commit/5d78c87a25fa430a2e39e7ba6d600f2edcf51c1f
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-10-27 (Thu, 27 Oct 2016)

  Changed paths:
M pkgs/development/interpreters/python/cpython/2.7/default.nix
M pkgs/development/python-modules/numpy.nix

  Log Message:
  ---
  numpy: enable numpy.distutils patch only if it's also in distutils

Fixes: 095095c ('python: add C++ compiler support for distutils')


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

Re: [Nix-dev] Custom libc

[Dmitry Kalinkin]

packageOverrides don’t override anything within stdenv

> On 12 Oct 2016, at 18:28, Tomasz Czyż  wrote:
> 
> Nikita, how do you override the attributes?
> 
> Did you try packageOverrides like in example below?
> 
> {
>  packageOverrides = pkgs: rec {
># Make "xbmc" use the "python26" package,
>#   instead of NixPkgs default python version.
>xbmc = pkgs.xbmc.override {
>  python = pkgs.python26;
>};
>  };
>}
> 
> 2016-10-12 22:16 GMT+01:00 Nikita Karetnikov :
> How can I override glibc in nix-shell and use eglibc, for instance? Are there 
> any examples? Do I need to override stdenv for that? Overriding the 
> attributes of glibc (version and sha) directly doesn't seem to work. Could 
> anyone provide an example?
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> 
> 
> 
> -- 
> Tomasz Czyż
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-commits] [NixOS/nixpkgs] 45b2a6: dejavu-fonts: enable on OSX

[Dmitry Kalinkin]

  Branch: refs/heads/release-16.09
  Home:   https://github.com/NixOS/nixpkgs
  Commit: 45b2a6d88299bb5671e36a4239ec0097bc149fb3
  
https://github.com/NixOS/nixpkgs/commit/45b2a6d88299bb5671e36a4239ec0097bc149fb3
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-10-06 (Thu, 06 Oct 2016)

  Changed paths:
M pkgs/data/fonts/dejavu-fonts/default.nix

  Log Message:
  ---
  dejavu-fonts: enable on OSX

(cherry picked from commit 780d9d64815d5f525e637930d051b40c1aeb5250)


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

[Nix-commits] [NixOS/nixpkgs] e7afac: ffmpeg3: fix darwin build (#18350)

[Dmitry Kalinkin]

  Branch: refs/heads/master
  Home:   https://github.com/NixOS/nixpkgs
  Commit: e7afac83c95c7ac03a1d82c5bdd94feebb44aec8
  
https://github.com/NixOS/nixpkgs/commit/e7afac83c95c7ac03a1d82c5bdd94feebb44aec8
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-09-06 (Tue, 06 Sep 2016)

  Changed paths:
M pkgs/development/libraries/ffmpeg/3.1.nix
A pkgs/development/libraries/ffmpeg/sdk_detection.patch

  Log Message:
  ---
  ffmpeg3: fix darwin build (#18350)


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

[Nix-commits] [NixOS/nixpkgs] 4449ec: root: 6.04.16 -> 6.04.18 (#17265)

[Dmitry Kalinkin]

  Branch: refs/heads/master
  Home:   https://github.com/NixOS/nixpkgs
  Commit: 4449ecb617a5d0bdc59898ccc7d246abbd10c359
  
https://github.com/NixOS/nixpkgs/commit/4449ecb617a5d0bdc59898ccc7d246abbd10c359
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-07-30 (Sat, 30 Jul 2016)

  Changed paths:
M pkgs/applications/science/misc/root/default.nix

  Log Message:
  ---
  root: 6.04.16 -> 6.04.18 (#17265)

root: compile against external gsl

Revert "Revert "root: enable parallel building""

This reverts commit ee1a10ee6b4b0a1a22e3e0dbbcf96a2d667d0685.

ROOT has no problems with parallel build. The problems that caused the
original commit were not related to parallel building.


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits

[Nix-commits] [NixOS/nix] 2a788b: documentation: mention required openssl dependency...

[Dmitry Kalinkin]

  Branch: refs/heads/master
  Home:   https://github.com/NixOS/nix
  Commit: 2a788bf9e7bc1ff4b464e9266bd490b61d9aa9e9
  
https://github.com/NixOS/nix/commit/2a788bf9e7bc1ff4b464e9266bd490b61d9aa9e9
  Author: Dmitry Kalinkin <dmitry.kalin...@gmail.com>
  Date:   2016-07-20 (Wed, 20 Jul 2016)

  Changed paths:
M doc/manual/installation/prerequisites-source.xml

  Log Message:
  ---
  documentation: mention required openssl dependency (#974)

Fixes: a6ca68a7 ("Require OpenSSL")


___
nix-commits mailing list
nix-comm...@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-commits