Arnold, Danylo, thank you for info, links and examples!
2017-03-15 9:17 GMT+00:00 Danylo Hlynskyi <[email protected]>: > Host side: > > networking.bridges.${bridgeName}.interfaces = []; > networking.interfaces.${bridgeName}.ip4 = [ { address = > "${subnet}.1"; prefixLength = 24; } ]; > > # Each container takes at least 4 inotify file handles, so you quickly > reach limit 128 when spawning many containers > boot.kernel.sysctl."fs.inotify.max_user_instances" = 2048; > > # internet in containers! > networking.nat.enable = true; > networking.nat.internalInterfaces = [ "ve-+" "vb-+" bridgeName ]; # > TODO check if bridgeName is required > networking.nat.externalInterface = "eth0"; > > Container side: > > privateNetwork = true; > hostBridge = bridgeName; > localAddress = container_ip + "/24"; > config = { > networking.defaultGateway = "${subnet}.1"; > networking.extraHosts = '' > ${subnet}.1 ${hostConfig.networking.hostName} > ''; > }; > > There is also some code to set-up hostname resolving via /etc/hosts, > because I know little about DHCP > , mDNS, nss-* and so on (https://github.com/NixOS/nixpkgs/issues/16754). > It is a wrapper around `containers` > and monitors IP clashes (which are set manually in my setup) > > > In general, I have 1 bridge with multiple IPs/subnets, so containers in > one subnet don't see containers in other. > > 2017-03-14 20:12 GMT+02:00 Tomasz Czyż <[email protected]>: > >> >> ---------- Forwarded message ---------- >> From: Tomasz Czyż <[email protected]> >> Date: 2017-03-14 18:12 GMT+00:00 >> Subject: Re: [Nix-dev] nixos-container networking >> To: Danylo Hlynskyi <[email protected]> >> >> >> Hey Danylo, >> >> yup, I hit 13 char limit and because I was removing - I thought it's that. >> >> I also had the issue with recreating containers, but this happened only >> sometimes and didn't happen when I restart the machine so I was not sure >> why is that. Thanks for your mail, it's very useful. >> >> Would you share your bridget networking? >> I was trying that but I'm wondering if you have one shared bridge or you >> have bridge per container and how you access containers from host (or route >> traffic to them). >> >> Cheers, >> Tom >> >> >> 2017-03-14 6:01 GMT+00:00 Danylo Hlynskyi <[email protected]>: >> >>> Strange, I have lot's of containers with "-" and experience no problems. >>> But maybe you've exceeded by accident limit 13 symbols per container name? >>> >>> Also, last time I tried "veth" networking, I was struggling from >>> https://github.com/NixOS/nixpkgs/issues/16330. My container experience >>> was awful when I tried container renames. That's why I've already switched >>> to bridged networking >>> >>> --- >>> >>> BTW, I highly recommend patch to switch-to-configuration.pl >>> <https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b277f78ece1bb81b79b5651897e46a2bf#diff-0a057d6ff3f6f83f68b859178484f4fe> >>> from https://github.com/NixOS/nixpkgs/pull/3021/commits/6e36619b2 >>> 77f78ece1bb81b79b5651897e46a2bf >>> >>> It isn't clear from commit message, but it does the following: makes >>> declarative containers truly reloadable (when you change >>> container config, it activates new configuration for container). The >>> culprit is *it should be* default behavior, because of >>> >>> 1. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/v >>> irtualisation/containers.nix#L225-L230 >>> 2. https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/v >>> irtualisation/containers.nix#L676 >>> >>> I'd like to PR this, but got no time to test properly other parts of >>> Nixos. >>> >>> 2017-03-14 4:42 GMT+02:00 Tomasz Czyż <[email protected]>: >>> >>>> Michael, Ian, thank you for your answers. >>>> >>>> Looks like my problem was with the container name. I tried bunch of >>>> different setups which didn't work and I discovered that when I'm using "-" >>>> in container name it doesn't work (I had impression that worked one or two >>>> times when I started machine from scratch, but most of the time didn't). >>>> >>>> After I removed "-" from the name, looks like private network is >>>> working (I can access private IP of container) so I don't need NAT >>>> actually. >>>> >>>> Tom >>>> >>>> 2017-03-13 23:54 GMT+00:00 Ian-Woo Kim <[email protected]>: >>>> >>>>> I've recently made nixos-container port forwarding easier (both >>>>> imperative and declarative) and it's now merged into master. >>>>> >>>>> https://github.com/NixOS/nixpkgs/pull/20869 >>>>> >>>>> Hope that this helps. >>>>> >>>>> Ian >>>>> >>>>> On Sun, Mar 12, 2017 at 7:52 PM, Michael Walker <[email protected]> >>>>> wrote: >>>>> > Tomasz, >>>>> > >>>>> > I have declarative container networking set up and working on a VPS, >>>>> > but I wrote most of the configuration as I was learning things, so it >>>>> > may not be the best way. >>>>> > >>>>> > Here's the configuration.nix for the VPS: >>>>> > https://github.com/barrucadu/nixfiles/blob/master/hosts/inns >>>>> mouth.nix >>>>> > Each container has a config file here: >>>>> > https://github.com/barrucadu/nixfiles/tree/master/containers >>>>> > >>>>> > Containers have ports forwarded to them via NAT; each container is >>>>> > running a web server on port 80 with the host reverse-proxying via >>>>> > nginx; the host also does https and letsencrypt for all the proxied >>>>> > containers. >>>>> > >>>>> > At the top of the innsmouth.nix file, I have a "containerSpecs" >>>>> record >>>>> > which has all the details for each container. The relevant bits of >>>>> the >>>>> > config are: >>>>> > >>>>> > 1. Set up the networking and NAT: >>>>> > >>>>> > networking.nat.enable = true; >>>>> > networking.nat.internalInterfaces = ["ve-+"]; >>>>> > networking.nat.externalInterface = "enp0s4"; >>>>> > >>>>> > 2. Forward ports to containers: >>>>> > >>>>> > networking.nat.forwardPorts = concatMap >>>>> > ( {num, ports, ...}: >>>>> > map (p: { sourcePort = p; destination = >>>>> > "192.168.255.${toString num}:${toString p}"; }) ports >>>>> > ) containerSpecs'; >>>>> > >>>>> > 3. Define all the containers: >>>>> > >>>>> > containers = mapAttrs >>>>> > (_: {num, config, ...}: >>>>> > { autoStart = true >>>>> > ; privateNetwork = true >>>>> > ; hostAddress = "192.168.254.${toString num}" >>>>> > ; localAddress = "192.168.255.${toString num}" >>>>> > ; config = config >>>>> > ; } >>>>> > ) containerSpecs; >>>>> > >>>>> > 4. Reverse-proxy HTTPS to HTTP in each container, manage letsencrypt >>>>> > certificates, and forward HTTP to HTTPS. >>>>> > >>>>> > This is a little complex as I have a fairly custom nginx config (see >>>>> > the services/nginx.nix file in the repository), but the >>>>> > reverse-proxying is fairly straightfoward. Here is the generated >>>>> > nginx.conf: https://misc.barrucadu.co.uk/nginx.txt >>>>> > >>>>> > On 13 March 2017 at 02:12, Tomasz Czyż <[email protected]> >>>>> wrote: >>>>> >> Hey, >>>>> >> >>>>> >> could anyone using nixos-container (declarative style) share how >>>>> you setup >>>>> >> networking? >>>>> >> >>>>> >> I'm trying to setup few containers with private network and http >>>>> proxy at >>>>> >> the front. Each container potentially could run application on port >>>>> 80 and I >>>>> >> would like to expose them through proxy. >>>>> >> >>>>> >> I tried to set this up with >>>>> >> >>>>> >> privateNetwork=true; >>>>> >> hostAddress >>>>> >> localAddress >>>>> >> >>>>> >> and I tried to also run nat on the host with (just to enable >>>>> outbound >>>>> >> traffic) >>>>> >> internalInterfaces = ["ve-+"]; >>>>> >> externalInterfaces = "eth0"; >>>>> >> >>>>> >> but no luck. >>>>> >> My next try will be creating bridge on the host and add containers >>>>> to that >>>>> >> bridge. Is that how you do stuff or are better ways of doing >>>>> container >>>>> >> networking? >>>>> >> >>>>> >> Tom >>>>> >> >>>>> >> _______________________________________________ >>>>> >> nix-dev mailing list >>>>> >> [email protected] >>>>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev >>>>> >> >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Michael Walker (http://www.barrucadu.co.uk) >>>>> > _______________________________________________ >>>>> > nix-dev mailing list >>>>> > [email protected] >>>>> > http://lists.science.uu.nl/mailman/listinfo/nix-dev >>>>> >>>> >>>> >>>> >>>> -- >>>> Tomasz Czyż >>>> >>>> _______________________________________________ >>>> nix-dev mailing list >>>> [email protected] >>>> http://lists.science.uu.nl/mailman/listinfo/nix-dev >>>> >>>> >>> >> >> >> -- >> Tomasz Czyż >> >> >> >> -- >> Tomasz Czyż >> >> _______________________________________________ >> nix-dev mailing list >> [email protected] >> http://lists.science.uu.nl/mailman/listinfo/nix-dev >> >> > -- Tomasz Czyż
_______________________________________________ nix-dev mailing list [email protected] http://lists.science.uu.nl/mailman/listinfo/nix-dev
