Re: [Nix-dev] Variable interpolation in strings read from files
On 06/25/2014 11:52 AM, Florian Friesdorf wrote:
Am I correct that my solution performs about as good as the in-place
testScripts for tests/installer.nix which us ${ } and that as Vlada said
this would be bad within pkgs.* but okish within nixos/tests?
I think so (from my previous skim).
Would caching of function values within nix be an option, so that
caching does not only happen via the store?
The value is cached within a single nix evaluation run. If we wanted to
cache expensive subexpression values between runs, we would need to
store them in nix store (or something similar)... that seems like a
quite complex project, although perhaps feasible.
Recursive nix might be a better way, as it's already implemented IIRC,
but there's still the inherent complexity of (the first) evaluation,
which is not nice and not expected by the current design.
Vlada
smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
Hi Eelco, Vlada,
thank you for your feedback!
Eelco Dolstra eelco.dols...@logicblox.com writes:
On 22/06/14 16:41, Florian Friesdorf wrote:
Personally, I would try to do all work from that substituteAll point in
derivations and not through nix strings (i.e. catenate by cat command).
Right. Doing large string rewrites at evaluation time is a bad idea
because it's slow.
I understand now that with pkgs.substiuteAll I would write a new file
which serves as an input to my expression and would only be updated if
the source file changed in contrast to do the substitution with every
evaluation.
Am I correct that my solution performs about as good as the in-place
testScripts for tests/installer.nix which us ${ } and that as Vlada said
this would be bad within pkgs.* but okish within nixos/tests?
Would caching of function values within nix be an option, so that
caching does not only happen via the store?
I've got a working solution:
https://github.com/chaoflow/nixpkgs/blob/2efc8fe32239cccab0e1b89b3c0983a2857dd128/nixos/tests/python.nix#L23
Should I move replaceSubstring and interpolateAtAtVars to
lib/strings.nix?
At first glance, these functions seem pretty slow to me (since they use
splitString). So it's probably not a good idea to add them to the standard
library (and I'd prefer removing splitString from there).
They are definitely slow in their current form.
Florian
--
Florian Friesdorf f...@chaoflow.net
GPG FPR: 7A13 5EEE 1421 9FC2 108D BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: f...@chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
pgpw179G8PqGF.pgp
Description: PGP signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
Hi, On 22/06/14 16:41, Florian Friesdorf wrote: Personally, I would try to do all work from that substituteAll point in derivations and not through nix strings (i.e. catenate by cat command). Right. Doing large string rewrites at evaluation time is a bad idea because it's slow. I've got a working solution: https://github.com/chaoflow/nixpkgs/blob/2efc8fe32239cccab0e1b89b3c0983a2857dd128/nixos/tests/python.nix#L23 Should I move replaceSubstring and interpolateAtAtVars to lib/strings.nix? At first glance, these functions seem pretty slow to me (since they use splitString). So it's probably not a good idea to add them to the standard library (and I'd prefer removing splitString from there). -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
On 06/22/2014 04:41 PM, Florian Friesdorf wrote: Should I move replaceSubstring and interpolateAtAtVars to lib/strings.nix? Oh, I forgot to reply. Basically my view is that of Eelco. The problem is that these things are run every time you try to install something (transitively) depending on it. Therefore it isn't fit for regular usage (especially in heavily used packages), although it's probably fine for separated tests, which is this case if I skim it correctly. Vlada ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
On 06/22/2014 01:11 PM, Florian Friesdorf wrote: It would be nice if there would be a way to interpolate variables in the string read from ./script.pl, just like the content would be written in-place. Usually one can use substituteAll; see qt4 for example. It uses a different syntax for what is replaced, but one typically does need a different syntax if substituting in shell-like files. Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
Upfront, I partly talked non-sense before, so I might have transported a wrong picture. Vladimír Čunát vcu...@gmail.com writes: On 06/22/2014 01:11 PM, Florian Friesdorf wrote: It would be nice if there would be a way to interpolate variables in the string read from ./script.pl, just like the content would be written in-place. Usually one can use substituteAll; see qt4 for example. It uses a different syntax for what is replaced, but one typically does need a different syntax if substituting in shell-like files. This is working using import and some nix boilerplate in the perl files. https://github.com/chaoflow/nixpkgs/blob/7482884953fcd8d3f29d84b864c53c46728d6910/nixos/tests/python.nix#L23 Now, I'd like to get rid of that nix boilerplate. It feels that a library function doing what substituteAll is doing would be great: testScriptPreamble = interpolate attrset (readFile ./python/preamble.pl); I'm happy to switch to whatever variable markup and also to write that function. Just don't want to go a direction that makes no sense. Any thoughts? -- Florian Friesdorf f...@chaoflow.net GPG FPR: 7A13 5EEE 1421 9FC2 108D BAAF 38F8 99A3 0C45 F083 Jabber/XMPP: f...@chaoflow.net IRC: chaoflow on freenode,ircnet,blafasel,OFTC pgpkVbJRbawxT.pgp Description: PGP signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
On 06/22/2014 02:55 PM, Florian Friesdorf wrote: Any thoughts? I *did* mean using the library function substituteAll. (It's not only a bash function in stdenv's setup.) It uses practically the syntax you propose. See qt4 for example. Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
Vladimír Čunát vcu...@gmail.com writes:
On 06/22/2014 02:55 PM, Florian Friesdorf wrote:
Any thoughts?
I *did* mean using the library function substituteAll. (It's not only a
bash function in stdenv's setup.) It uses practically the syntax you
propose. See qt4 for example.
qt4 uses pkgs.substituteAll to read a source file, perform substitution
on it and write a target file. It returns the path to the target file.
This works and I get a file with variables substituted, e.g.:
/nix/store/x32hziqm127agl1ipr6sfs7p9j8zdvpm-preamble.pl
Now I try to read this file:
testScriptPreamble = { debug, full, pythonstr }:
let debug_ = if debug then true else ;
full_ = if full then true else ;
in readFile (pkgs.substituteAll { src = ./python/preamble.pl;
inherit debug_ full_ pythonstr; });
and get:
% nix-build python.nix -A virtualenvPython27Full
error: string `/nix/store/x32hziqm127agl1ipr6sfs7p9j8zdvpm-preamble.pl' cannot
refer to other paths, at /home/cfl/dev/nixos/nixpkgs/nixos/tests/python.nix:26:8
Current full experiment:
https://github.com/chaoflow/nixpkgs/blob/5f2341e96cd48c6d75ca9abf7447cc4b04f04611/nixos/tests/python.nix#L26
Is there a way to get this work with pkgs.substituteAll?
I'm thinking of something like:
testScriptPreamble = { debug, full, pythonstr }:
let attrs = {
debug_ = if debug then true else ;
full_ = if full then true else ;
inherit pythonstr;
};
in pkgs.lib.substitute attrs (readFile ./python/preamble.pl);
--
Florian Friesdorf f...@chaoflow.net
GPG FPR: 7A13 5EEE 1421 9FC2 108D BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: f...@chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
pgpaZWpxUqqtc.pgp
Description: PGP signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
On 06/22/2014 03:45 PM, Florian Friesdorf wrote: Now I try to read this file: Ah, sorry, I didn't notice that you used the result as a nix string. Switching back and forth between nix (strings) and derivations (files) is better avoided. The readFile function is only meant for what you have among the nix source expressions. One probably can do more with recursive nix. Personally, I would try to do all work from that substituteAll point in derivations and not through nix strings (i.e. catenate by cat command). Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Variable interpolation in strings read from files
Vladimír Čunát vcu...@gmail.com writes: On 06/22/2014 03:45 PM, Florian Friesdorf wrote: Now I try to read this file: Ah, sorry, I didn't notice that you used the result as a nix string. Switching back and forth between nix (strings) and derivations (files) is better avoided. The readFile function is only meant for what you have among the nix source expressions. One probably can do more with recursive nix. Personally, I would try to do all work from that substituteAll point in derivations and not through nix strings (i.e. catenate by cat command). I've got a working solution: https://github.com/chaoflow/nixpkgs/blob/2efc8fe32239cccab0e1b89b3c0983a2857dd128/nixos/tests/python.nix#L23 Should I move replaceSubstring and interpolateAtAtVars to lib/strings.nix? Suggestions for better naming? -- Florian Friesdorf f...@chaoflow.net GPG FPR: 7A13 5EEE 1421 9FC2 108D BAAF 38F8 99A3 0C45 F083 Jabber/XMPP: f...@chaoflow.net IRC: chaoflow on freenode,ircnet,blafasel,OFTC pgpgCPZ4wOU6C.pgp Description: PGP signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
