Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-03-04 Thread Vladimír Čunát 
On 03/04/2017 05:30 PM, Jan Malakhovski wrote:
> Thirdly, I don't ever use `nixos-rebuild`, because most of the time I
> produce derivations locally on a laptop and then push builds to other
> machines near the machines the results belong to (which `nixos-rebuild`
> can't do, AFAIK).

For reference, there are newer parameters to nixos-rebuild: --build-host
and --target-host

--Vladimir




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-03-04 Thread Jan Malakhovski 
Hi.

I seem to be doing something different from all the comments, so I
though I'd share.

Firstly, I maintain a huge number of machines from a single place and
run many builds at once (and switch between nixpkgs branches a lot), so
all the symlinking doesn't cut it.

Secondly, I don't use nixops, because my configs require more flex
(nixops `hardware/software` split doesn't cut it for me, I use
`hardware/network-env/software/temp-stuff` configs for machines and
manage most of that through environment variables).

Thirdly, I don't ever use `nixos-rebuild`, because most of the time I
produce derivations locally on a laptop and then push builds to other
machines near the machines the results belong to (which `nixos-rebuild`
can't do, AFAIK). Most of the time I don't even trust those machines
enough to send all of my nix files to them. Managing subsets of files to
rsync/branches to git-push to various hosts is painful. So I just store
everything in a central place and only push derivations around.

For instance, say, I'm Elon Musk, and I have `slow-pc` machine near
`fast-build-cluster` on Mars and I want to update `slow-pc`'s software,
but I don't trust any of the two machines not to be hijacked by aliens
with (general) quantum computers that would break my public keys used
for my machines on Earth if they got their hands on key material. That,
of course, assumes aliens haven't hijacked (or got in bed with enough
people from) NSA/FiveEyes yet and can't sniff my pubkeys from undersea
cables here on Earth (or maybe I decided to spend some more billions for
the future of Humanity and build my own
non-compromised-by-government-for-your-money Internet infrastructure).
Anyway, I trust my skills not to leak keys in unrelated derivations/not
paranoid enough to setup completely separate repo for machines on Mars.

With my build.sh script I do

  $ build.sh machine.slow-pc --on fast-build-cluster.on.mars --for 
slow-pc.on.mars --and-then switch

and go back to worrying about Earthly matters.

Actually, most of the time I do
  
  $ build.sh machine.slow-pc --on fast-build-cluster.on.mars --tmux

and then detach from tmux, go for a walk, attach back, and only then run
the previous command, if everything seems fine. The script can be
enhanced in that regard, I know, but I'm lazy.

Some other uses:

* Rebuild laptop's config on a trusted host near me

  $ build.sh machine.me --on trusted-fast-build-cluster.near.me --for . 
--and-then boot

* Build manpages for the machine

  $ build.sh doc.slow-pc 

* Build /etc

  $ build.sh etc.slow-pc 

* Build package `qemu` with overrides from `slow-pc`s config

  $ build.sh mpackage.slow-pc.qemu

* Just a wrapper around the usual `nix-build -A qemu`

  $ build.sh package.qemu

See attachments for the scripts and patches to nixpkgs required (I hope
I haven't forgot anything, my private branch is >200 commits atm).

As to the NixOS configs I use a combination of nixos and nixops
approaches: each machine has its own `machine/.nix` root which
requires all the common things, including my own `modules` that threat
`nixos/modules` as `nixos/modules` treat nixpkgs.

I.e. I have my own set of NixOS subtrees of options. E.g.

  oxij.role.mailer = {
enable = true;
parent = "some-host.near.me";
  };

in `machine/.nix configures postfix/dovecot/spam filter/etc in a
single go.

For instance, config of my machine looks like this:

  # Some special snow-flaking
  oxij.hardware.me = true;
  oxij.hardware.xen = true;

  # This adds users used on all home machines with relevant pubkeys,
  # sets up network config, configures the use of common tor/i2p
  # instances on home gateway (so that traffic from different machines
  # won't be distinguishable from outside and to use other machines for
  # cover traffic) and so on.
  # This module actually pattern matches on hostname inside to disable
  # access to some users on some machines (but add this users anyway so
  # that `ls` would print names instead of ids).
  oxij.location.home = true;

  # Special user for giving public presentations and stuff. Privacy
  # matters, right?
  oxij.location.public = true;

  # developers! Developers! DEVELOPERS!
  oxij.role.dev.all = true;
  oxij.role.dev.js = mkForce false;
  oxij.role.tex = true;
  oxij.role.emacs = true;
  oxij.role.xmonad = true;
  oxij.role.desktop = true;

Also, I allow overriding much of the stuff with environment variables
(unlike nixops) instead of using separate files for every singe build
option.

For instance, my machine has several other modes of operation, e.g.
running

  $ MACHINE_PROFILE=roaming build.sh machine.me

adds

  # Build our own tor/i2p routers and provide no external ssh access for
  # "out of home" experience
  oxij.location.home = mkForce false;
  oxij.location.work = mkForce false;
  oxij.location.roaming = true;

and

  $ MACHINE_PROFILE=work build.sh machine.me

does

  # Like roaming, but configure some work stuff
  oxij.location.home = mkForce false;

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-03-01 Thread Badi' Abdul-Wahid 
My approach mirrors Jookia's as well. I put all my configuration files into
a single repository and make /etc/nixos/configuration.nix look something
like:

{ imports = [ /home/badi/nixos/fangorn.nix ]; }

I found symlinking configuration.nix to the appropriate entrypoint
problematic when moving between machines (in addition to being on github,
the repo is also synced directly between each node using syncthing. Dropbox
or some other shared filesystem could work as well).

Associated configuration sets are in separate files such as users.nix,
monitoring.nix, syncthing.nix, basicsystem.nix, laptop.nix, etc, while
machine specific options are defined in .nix (eg fangorn.nix in
above), which then imports the various options as needed.

This works fairly well as I routinely move between several machines that I
want to keep as identical as possible.

On Mon, Feb 27, 2017 at 9:08 PM, Profpatsch  wrote:

> On 17-02-27 09:15am, Mark Gardner wrote:
> > Now that I am putting NixOS on more and more machines, I would like to
> > modularize and share parts of the config to maximize reuse and ensure
> > uniformity.
>
> As far as the non-dev-ops-y parts go
> (that is: the nix expression parts),
> there’s basically two ways to structure modules:
>
> 1. Using `import`
> 2. Using `imports` from the module system
> 3. Using option definitions
>
> All have their place, I personally use the first
> two for more ephemeral settings and the latter if
> I want to share something for use by others.
>
> Example from my current machine config in our vuizvui repo:
>
> I have a folder for my personal machines, with a lib.nix
> and pkgs.nix; these are included as attrsets with `import`
> https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463
> b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L4
>
> My config for workstation/server/base is split into various
> module files, which get included with `imports`.
> https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463
> b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L9
> https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463
> b7d31dad6bfc95f01008c787/machines/profpatsch/base-workstation.nix#L11
>
> Then finally there’s the Thinkpad-specific setup,
> which I have put into a module with an enable option
> for others to use.
> https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463
> b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L45
> https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463
> b7d31dad6bfc95f01008c787/modules/hardware/thinkpad.nix
> and it’s actually used by our hackerspace’s Thinkpad, too:
> https://github.com/openlab-aux/vuizvui/blob/cd4bc6fde00e3da0
> 382a3eb151dbbd0087449377/machines/labnet/labtops.nix#L9
>
> I hope that helps.
>
>
> --
> Proudly written in Mutt with Vim on NixOS.
> Q: Why is this email longer than five sentences?
> A: Sometimes you have to be more verbose. :)
> May take up to five days to read your message. If it’s urgent, call me.
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>



-- 

Badi' Abdul-Wahid
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread Profpatsch 
On 17-02-27 09:15am, Mark Gardner wrote:
> Now that I am putting NixOS on more and more machines, I would like to
> modularize and share parts of the config to maximize reuse and ensure
> uniformity. 

As far as the non-dev-ops-y parts go
(that is: the nix expression parts),
there’s basically two ways to structure modules:

1. Using `import`
2. Using `imports` from the module system
3. Using option definitions

All have their place, I personally use the first
two for more ephemeral settings and the latter if
I want to share something for use by others.

Example from my current machine config in our vuizvui repo:

I have a folder for my personal machines, with a lib.nix
and pkgs.nix; these are included as attrsets with `import`
https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L4

My config for workstation/server/base is split into various
module files, which get included with `imports`.
https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L9
https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463b7d31dad6bfc95f01008c787/machines/profpatsch/base-workstation.nix#L11

Then finally there’s the Thinkpad-specific setup,
which I have put into a module with an enable option
for others to use.
https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463b7d31dad6bfc95f01008c787/machines/profpatsch/katara.nix#L45
https://github.com/openlab-aux/vuizvui/blob/ac8da8796649e463b7d31dad6bfc95f01008c787/modules/hardware/thinkpad.nix
and it’s actually used by our hackerspace’s Thinkpad, too:
https://github.com/openlab-aux/vuizvui/blob/cd4bc6fde00e3da0382a3eb151dbbd0087449377/machines/labnet/labtops.nix#L9

I hope that helps.


-- 
Proudly written in Mutt with Vim on NixOS.
Q: Why is this email longer than five sentences?
A: Sometimes you have to be more verbose. :)
May take up to five days to read your message. If it’s urgent, call me.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread ben...@gmail.com 
My approach is quite similar to Jookia's.  I also point nixpkgs at a
remote tarball from a github branch that I control, which takes the
place of using nix-channel, and set nixos-config relative to
config.networking.hostName.  With this setup, I don't need any
symlinks, nor custom imports in ~/.nixpkgs/config.nix because it has
the same view of nixpkgs as the rest of the system.

In nix.nix[1], imported from configuration.nix[2]:

nix.nixPath = [
  "nixpkgs=https://github.com/benley/archive/nixos-benley.tar.gz;
  
"nixos-config=/home/benley/p/dotfiles/machines/${config.networking.hostName}/configuration.nix"
  "dotfiles=/home/benley/p/dotfiles"
];

In ~/.nix-defexpr/nixpkgs.nix:

import  { }

Having  be an element of NIX_PATH has also been useful in
various places, such as in ~/default.nix, which is used by
nix-home[3].

In ~/default.nix:

(import  { }).homedir

And finally I have a directory full of dotfiles that get overlaid into
my homedir, for various things that don't really belong in the
system-wide config[4].  This works equally well with non-nixos
systems, as long as they have nix installed.

In theory I could point nixos-config at a github URL too, but I like
the convenience of being able to directly edit it in my homedir as
needed.

[1]: https://github.com/benley/dotfiles/blob/master/machines/imports/nix.nix
[2]: 
https://github.com/benley/dotfiles/blob/master/machines/samus/configuration.nix
[3]: https://github.com/sheenobu/nix-home
[4]: https://github.com/benley/dotfiles/tree/master/cfg (also see
homedir.nix in this repo)

On Mon, Feb 27, 2017 at 3:40 PM, Jookia <166...@gmail.com> wrote:
>
> On Mon, Feb 27, 2017 at 09:15:27AM -0500, Mark Gardner wrote:
> > So far, this seems like a good approach. Except that each machine has its
> > own configuration.nix that I would like to keep in the git repository too
> > but of course I can't have different top level files with the same name. To
> > solve this, I could moved the current configuration.nix inside of cfg (as
> > cfg/mylaptop.cfg.nix perhaps) or merge with the existing cfg/mylaptop.nix
> > then making configuration.nix a symlink to it. That way the only thing to
> > do by hand is create the symlink to select a particular configuration. Is
> > this reasonable? Is there a better way to do it?
> >
> > How do you modularize your configuration and put it into a repo such that
> > you can easily create a configuration for a new machine (and put it in the
> > repo too) without a lot of hand work?
> >
> > Mark
> > --
> > Mark Gardner
> > --
>
> I have a configuration for each host, but then I set my nix path like so:
>
> NIX_PATH=nixpkgs=/home/jookia.data/nixos/nixpkgs.git:nixos-config=/home/jookia.data/nixos/configuration.git/novena.nix
>
> nixos-config points to my host-specific configuration. Then in that host
> configuration I set the path something like this:
>
> nix.nixPath = [
>   "nixos-config=/home/jookia.data/nixos/configuration.git/novena.nix"
> ];
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread Jookia 
On Mon, Feb 27, 2017 at 09:15:27AM -0500, Mark Gardner wrote:
> So far, this seems like a good approach. Except that each machine has its
> own configuration.nix that I would like to keep in the git repository too
> but of course I can't have different top level files with the same name. To
> solve this, I could moved the current configuration.nix inside of cfg (as
> cfg/mylaptop.cfg.nix perhaps) or merge with the existing cfg/mylaptop.nix
> then making configuration.nix a symlink to it. That way the only thing to
> do by hand is create the symlink to select a particular configuration. Is
> this reasonable? Is there a better way to do it?
> 
> How do you modularize your configuration and put it into a repo such that
> you can easily create a configuration for a new machine (and put it in the
> repo too) without a lot of hand work?
> 
> Mark
> -- 
> Mark Gardner
> --

I have a configuration for each host, but then I set my nix path like so:

NIX_PATH=nixpkgs=/home/jookia.data/nixos/nixpkgs.git:nixos-config=/home/jookia.data/nixos/configuration.git/novena.nix

nixos-config points to my host-specific configuration. Then in that host
configuration I set the path something like this:

nix.nixPath = [
  "nixos-config=/home/jookia.data/nixos/configuration.git/novena.nix"
];
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread Vladimír Čunát 
Hi!

On 02/27/2017 03:15 PM, Mark Gardner wrote:
> - cfg/common.nix  # common config
> - cfg/desktop.nix  # xorg and related
> - cfg/laptop.nix  # related to all laptops

I have exactly those three imports, and I also symlink configuration.nix :-)

One more thing I enjoy:
  nixpkgs.config = import ./nixpkgs-config.nix;

so that I can also import it in ~/.nixpkgs/config.nix and have nix-env +
nix-build use the same pre-defined package-sets, overrides, and other
settings.

--Vladimir




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread David Izquierdo 
My setup is also very similar. I have a configuration.nix with all the 
common setup, which imports hosts/current.nix, which is a (.gitignored) 
symlink to the appropiate host.nix. Then, the host.nix imports from 
modules/*.nix. I also have an etc with generic (not written in Nix) 
configuration files that get imported via builtins.readFile (shell RCs, 
for example) or referred to by *.configFile (i3 config).



I'm planning a slight refactor in which the symlink is actually 
configuration.nix, so that I can make certain reusable declarations into 
a module (I like specific users, UIDs, passwords, and hostnames for each 
host, right now there's a lot of identical lines, varying only the 
username).



I _do_ sync the config tree to my server, since it also serves as a 
general purpose remote computer, and I end up using it a lot when not on 
a machine I own.



On 27/02/17 15:28, Tomasz Czyż wrote:

Hey Mark,

I use almost the same setup and for 1.5y works very well.

I have "modules" and "hosts" directories. Each "host" contains
configuration about hardware/disk setup and includes set of modules from
"modules". On each host there is a symlink to correct
"hosts/.nix" file.

I found this setup pretty robust on dev machines.
On servers I use nixops which pushes configuration so I don't manage
/etc/nixos there.

Tom

2017-02-27 14:15 GMT+00:00 Mark Gardner :


Now that I am putting NixOS on more and more machines, I would like to
modularize and share parts of the config to maximize reuse and ensure
uniformity. My approach is to consider the sub-config files as traits or
roles and combine them together to create configuration.nix for a specific
machine, like this:

- cfg/common.nix  # common config
- cfg/desktop.nix  # xorg and related
- cfg/laptop.nix  # related to all laptops
- cfg/work.nix  # work location related
...
- cfg/mylaptop.nix  # specific laptop related

I import from these to make up configuration.nix. For example, on my
laptop, configuration.nix contains:

---
{ config, pkgs, ... }:

{
   imports =
 [
   ./hardware-configuration.nix
   ./cfg/mylaptop.nix
   ./cfg/common.nix
   ./cfg/desktop.nix
   ./cfg/laptop.nix
   ./cfg/work.nix
 ];
}
---

So far, this seems like a good approach. Except that each machine has its
own configuration.nix that I would like to keep in the git repository too
but of course I can't have different top level files with the same name. To
solve this, I could moved the current configuration.nix inside of cfg (as
cfg/mylaptop.cfg.nix perhaps) or merge with the existing cfg/mylaptop.nix
then making configuration.nix a symlink to it. That way the only thing to
do by hand is create the symlink to select a particular configuration. Is
this reasonable? Is there a better way to do it?

How do you modularize your configuration and put it into a repo such that
you can easily create a configuration for a new machine (and put it in the
repo too) without a lot of hand work?

Mark
--
Mark Gardner
--

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev






___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread Tomasz Czyż 
Hey Mark,

I use almost the same setup and for 1.5y works very well.

I have "modules" and "hosts" directories. Each "host" contains
configuration about hardware/disk setup and includes set of modules from
"modules". On each host there is a symlink to correct
"hosts/.nix" file.

I found this setup pretty robust on dev machines.
On servers I use nixops which pushes configuration so I don't manage
/etc/nixos there.

Tom

2017-02-27 14:15 GMT+00:00 Mark Gardner :

> Now that I am putting NixOS on more and more machines, I would like to
> modularize and share parts of the config to maximize reuse and ensure
> uniformity. My approach is to consider the sub-config files as traits or
> roles and combine them together to create configuration.nix for a specific
> machine, like this:
>
> - cfg/common.nix  # common config
> - cfg/desktop.nix  # xorg and related
> - cfg/laptop.nix  # related to all laptops
> - cfg/work.nix  # work location related
> ...
> - cfg/mylaptop.nix  # specific laptop related
>
> I import from these to make up configuration.nix. For example, on my
> laptop, configuration.nix contains:
>
> ---
> { config, pkgs, ... }:
>
> {
>   imports =
> [
>   ./hardware-configuration.nix
>   ./cfg/mylaptop.nix
>   ./cfg/common.nix
>   ./cfg/desktop.nix
>   ./cfg/laptop.nix
>   ./cfg/work.nix
> ];
> }
> ---
>
> So far, this seems like a good approach. Except that each machine has its
> own configuration.nix that I would like to keep in the git repository too
> but of course I can't have different top level files with the same name. To
> solve this, I could moved the current configuration.nix inside of cfg (as
> cfg/mylaptop.cfg.nix perhaps) or merge with the existing cfg/mylaptop.nix
> then making configuration.nix a symlink to it. That way the only thing to
> do by hand is create the symlink to select a particular configuration. Is
> this reasonable? Is there a better way to do it?
>
> How do you modularize your configuration and put it into a repo such that
> you can easily create a configuration for a new machine (and put it in the
> repo too) without a lot of hand work?
>
> Mark
> --
> Mark Gardner
> --
>
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>


-- 
Tomasz Czyż
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] Best Practices on Modularizing Configuration.nix?

2017-02-27 Thread Mark Gardner 
Now that I am putting NixOS on more and more machines, I would like to
modularize and share parts of the config to maximize reuse and ensure
uniformity. My approach is to consider the sub-config files as traits or
roles and combine them together to create configuration.nix for a specific
machine, like this:

- cfg/common.nix  # common config
- cfg/desktop.nix  # xorg and related
- cfg/laptop.nix  # related to all laptops
- cfg/work.nix  # work location related
...
- cfg/mylaptop.nix  # specific laptop related

I import from these to make up configuration.nix. For example, on my
laptop, configuration.nix contains:

---
{ config, pkgs, ... }:

{
  imports =
[
  ./hardware-configuration.nix
  ./cfg/mylaptop.nix
  ./cfg/common.nix
  ./cfg/desktop.nix
  ./cfg/laptop.nix
  ./cfg/work.nix
];
}
---

So far, this seems like a good approach. Except that each machine has its
own configuration.nix that I would like to keep in the git repository too
but of course I can't have different top level files with the same name. To
solve this, I could moved the current configuration.nix inside of cfg (as
cfg/mylaptop.cfg.nix perhaps) or merge with the existing cfg/mylaptop.nix
then making configuration.nix a symlink to it. That way the only thing to
do by hand is create the symlink to select a particular configuration. Is
this reasonable? Is there a better way to do it?

How do you modularize your configuration and put it into a repo such that
you can easily create a configuration for a new machine (and put it in the
repo too) without a lot of hand work?

Mark
-- 
Mark Gardner
--
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev