Re: [Nix-dev] In multi-user Nix, let the daemon handle creation of GC roots
On 18/06/17 20:16, sba...@catern.com wrote: > Kevin Cox writes: >> On 18/06/17 03:49, sba...@catern.com wrote: >>> >> That being said I'm not sure this is necssary. You can already >> inspect/delete roots by a given user to see how much space they are >> "using". > > Yes, but that's a post-hoc check which can't really be made to work > reliably. What do you do if a user is using too much space? Delete their > stuff without any warning? That's unworkable, instead there should be an > error at the time that the user tries to consume excess > space. (Admittedly the policy around this will be tricky, but currently > we don't even have the mechanism to write such policy). So are you also going to stop the user from building new things? You would also have the problem if the user was inside their quota then started building something very large. I guess it wouldn't be bad to start allowing this sort of enforcement but I'm not sure how useful it would be in practice. > >> I'm curious what you would want to do to help with containers, >> or to help with putting nix on network storage. I don't see how this is >> required. > > It helps because it allows moving away from user-based control over GC > roots. Imagine two separate independent services running as the same > user in containers built with Nix. The two services have to run as the > same user because we don't have privileges on the system to create new > users. Those two services shouldn't be allowed to remove each others' GC > roots or add GC roots using each others' credentials. We should be able > to give the services conditional access to creating GC roots, without > full access to a per-user directory; we can do that if GC roots are > created through IPC. If you have two requests from the same user how do you segment them into container or some other division? If you can restrict the containers in some way couldn't you do this yourself by putting the links somewhere the container can't access? > > A similar issue exists for multiple hosts sharing a store. Those hosts > shouldn't need to have the exact same set of users, just to share a > store. I should be able to publish my store so it is publicly accessible > to the world at large, and then allow people to connect in > (authenticating in some way) to the nix-daemon managing that store and > register new GC roots so they can maintain deployments out of that > global store. This is a lot easier than requiring the use of NFS with > POSIX user accounts which authenticate via Kerberos (or don't > authenticate at all, in traditional NFS). > > The basic issue is tying control over GC roots to Unix user accounts. > Per-user software deployment is great and a big advance over root-only > software deployment, but Unix users have two fatal practical flaws: They > require privileges to create and they don't straightforwardly work > across the network. This hurts using a single store for multiple > containers on a single box, as well as using a single store across an > untrusting network. > > And the more fundamental issue is that Unix users aren't compatible with > a capability-security model, which hopefully Nix can help push the world > towards. I would like to be able to give out a capability for creating > GC roots; that's not possible with the current user-based mechanism. > > Hopefully that makes sense? I am fairly new to Nix, but I'm coming at > this from a capability-security mindset, and want to make sure Nix is as > compatible as possible with cap-security. > Same question as above, these sound like great goals but I don't see how the Nix daemon is going to be able to divide the requests according to capability. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] In multi-user Nix, let the daemon handle creation of GC roots
On 18/06/17 03:49, sba...@catern.com wrote: > > It seems to me that it would be useful for the daemon to handle making > GC roots, and forbid users to directly create GC roots. > > The features that come to mind: > - Allows later implementing policy about GC roots/space consumption > - Allows avoiding complicated locking around doing GC > - Allows /nix to be put on network storage transparently > - Allows /nix to be shared between containers transparently I'm not sure just this will solve your problem, however it might be a good start. For example if the daemon only created indirect roots then each root is accounted for (based on where it points) and can be cleaned up by either the system or the user. That being said I'm not sure this is necssary. You can already inspect/delete roots by a given user to see how much space they are "using". I'm curious what you would want to do to help with containers, or to help with putting nix on network storage. I don't see how this is required. Basically I don't have any fanstic reason this shouldn't be done (except simplicity) however I also don't really understand any of your arguments for why it should be done. I think some sort of design-document for solving one (or more) of these problems would be useful. This way there is a concrete example of why the current situation wouldn't work and provide a place for people to suggest concrete alternatives and debate the merits. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] still waiting for https://cache.nixos.org after 5 seconds...
On 13 May 2017 01:14, "Jookia" <166...@gmail.com> wrote: On Mon, May 08, 2017 at 11:46:02AM +, Denis wrote: > I really do not understand the reasons of the strong opposition to > another mirror on Cloudflare (free of cost, although it may not solve > the problem completely - it has no endpoint in Vietnam, for example - > it may increase availability and reduce Amazon bills) and to allowing > the people in regions to host mirrors (it should not be a security > breach as the packages are cryptographic signed). It wouldn't be great for proxy users who get blocked by CAPTCHAs You can turn that off. ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] nix rebuilds
On 05/12/2017 01:23 PM, Harmen wrote: I'm building a few ruby and node projects with nix. It works well enough, but nix does too many rebuilds, also when nothing changed. The question is: how I can figure out why it does a rebuild? Maybe some checksum has changed, but can I somehow figure out which one? I suspect that at a bare minimum the ./result symlink has changed. What I tend to do is use builtins.filterSource in order to ensure I am only including paths that I need. You can see an example at https://github.com/kevincox/sog/blob/master/default.nix src = builtins.filterSource (name: type: (lib.hasPrefix (toString ./BUILD) name) || (lib.hasPrefix (toString ./sog) name) || (lib.hasPrefix (toString ./test) name) || (lib.hasPrefix (toString ./tools) name) || (lib.hasPrefix (toString ./WORKSPACE) name)) ./.; ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] NixOS and nix 1.12
On 09/05/17 21:03, Vladimír Čunát wrote: > On 05/09/2017 12:46 PM, Tomasz Czyż wrote: >> Actually, wouldn't be great if we transfer that functions over a time >> rather than doing big bang? > > I'm not really involved in the development of Nix, but I don't think the > old interface will go away with 1.12 already. So far it's been a > separate "nix" command that didn't clash with the rest, I think, and I > don't know any reason to remove the old CLI too quickly. > I agree, I think it would be best to keep the old commands as-is, and slowly add functionality to the new command. I think that mapping the old commands into the new one just creates a second inconsistent method to call them that we will also have to eventually discourage. Doing it this way keeps only two interfaces, the old insonsistent ones (nix-*), and the new consistent and well thought out one (nix). signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] NixOps prefer send from local.
On 28/04/17 21:35, Danylo Hlynskyi wrote: > did you consider > > https://nixos.org/nixops/manual/#opt-deployment.hasFastConnection > Thanks, that's exactly what I was looking for. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] NixOps prefer send from local.
Hi all. I am using NixOps to configure a machine on the local network, unfortunately my uplink to the internet isn't great but the network is fast. Unfortunately when deploying I first download all the packages to my machine, then the remote host prefers using configured binary caches to download the packages. I was wondering if there is a way to prefer copying files from the local machine rather then consulting caches at all. I can remove the caches from the remote machines configuration but I do want to be able to install packages from the caches in user profiles, so while this is a workaround while deveoping I would need to remember to enable them again before I was done. I can also nix-serve the local machine but I am back to modifying the list of caches between deploying and using. So while I have two workarounds I was wondering if there was a simple solution. Thanks, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] What are the rules about 'share/' directory symlinking to profile?
On 20/04/17 10:46, Alexey Lebedeff wrote: Hi folks. What are the rules about how and when 'share/' folder of (multi-output) derivation will be merged into 'share/' folder of profile? E.g. I can't see content of https://github.com/NixOS/nixpkgs/blob/4a0be7837cadf9b6e9cae430d80f65a2af60dcc6/pkgs/tools/misc/fzf/default.nix#L33 anywhere in my profile 'share/' even after installing every output Only when I move 'share/' to '$bin' output, it starts appearing in profile 'share/' folder. There is a list of hard-coded directories to merge. I have been thinking about making it merge all directories but am not decided this is ideal since it is also nice to have "private" data that doesn't get "installed". ___ nix-dev mailing list nix-dev@lists.science.uu.nl https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] 'nixos-stable' channel?
On 06/03/17 06:54, Sander wrote: Moving a symlink twice a year? Good point. I guess it would be worth trying out and we can see how many people use this. I was thinking something more rolling would be nice too. Like a lightly tested. Although that would definitely have more maintenance costs. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] 'nixos-stable' channel? (was: Re: NixOS 17.03 Beta, 16.09 Security Support Timeline)
On Mar 5, 2017 19:26, "Bjørn Forsman" wrote: As long as the nixos-stable channel is an opt-in, why not? Well there would be a maintenance overhead. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] 'nixos-stable' channel? (was: Re: NixOS 17.03 Beta, 16.09 Security Support Timeline)
Oops, forgot to include the list. On Mar 5, 2017 16:15, "Sander" wrote: Graham Christensen wrote (ao): > NixOS 17.03 has entered Beta. This means we now have 3 versions of NixOS > being developed: > > - 16.09 (stable) > - 17.03 (beta) > - unstable Would it make sense to have a 'nixos-stable' channel that points to whatever channel is stable? It's kinda a weird concept, as there would be breaking changes whenever it switched between releases but I do see the use of a more tested stream then nixos-unstable. Maybe it would be a channel with breaking changes but "no" broken packages? I would love to hear what people think about this. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Auto upgrade
I don't use autoupgrade on my servers but quite often do manual upgrades and deploys without much safety checks and it usually seems to work out okay. The nice this about Nix is that you can always boot into an old version of the system from the bootloader which makes this a much safer proposition then autouograde on something like Ubuntu. That being said Nix doesn't (yet 😉) have a thorough QA system so it likely won't be as stable as Ubuntu in that respect. On Feb 24, 2017 08:06, "Mark Gardner" wrote: > How stable is my system likely to be if I set "system.autoUpgrade.enable = > true;" if I am on the stable channel (16.09)? How likely will a it be that > an upgrade will cause problems? Do I even need to worry? > > Mark > -- > Mark Gardner > -- > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-commits] [NixOS/nixpkgs] da33c8: systemd: Properly escape environment options.
Branch: refs/heads/master Home: https://github.com/NixOS/nixpkgs Commit: da33c8a19d97711d213789cf49c8b7c483beb18f https://github.com/NixOS/nixpkgs/commit/da33c8a19d97711d213789cf49c8b7c483beb18f Author: Kevin Cox Date: 2017-02-20 (Mon, 20 Feb 2017) Changed paths: M nixos/modules/system/boot/systemd.nix Log Message: --- systemd: Properly escape environment options. Using toJSON on a string value works because the allowed JSON escape sequences is almost a subset of the systemd allowed escape sequences. The only exception is `\/` which JSON allows but systemd doesn't. Luckily this sequence isn't required and toJSON don't produce it making the result valid for systemd consumption. Examples of things that this fixes are environment variables with double quotes or newlines. ___ nix-commits mailing list nix-comm...@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-commits
Re: [Nix-dev] Python 3 as default
On Feb 14, 2017 07:49, "Freddy Rietdijk" wrote: It would two mean things: 1. alias `python3 to `python` 2. add a symlink that points to the `python` program, as discussed before in this thread. For what it is worth I would prefer that we have a python executable in the path (if it is "installed"), but the nix expressions only have the version suffixed attributes to increase explicitness and improve readability. Plus then we will be ready for python 4 😛 ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Hydra Building PRs
That's excellent to get the integration tests running! As for the comment. If it's not too much work it would be great to integrate it with the status API (I think that's what it is called). It is much tydier and it's nice to get the pass/fail status highlighted in the UI. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Hydra Building PRs
On Jan 28, 2017 13:21, "Graham Christensen" wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hello everyone, A couple weeks ago I created https://prs.nix.gsc.io/, a Hydra server which builds every PR. Every PR will exist at https://prs.nix.gsc.io/jobset/nixos/pr-PRNUMBER. It features: - a web-hook, which means: - the moment a PR is created, the jobset is created on this hydra - every time a PR is updated, the jobset is triggered - every time there is a commit to master, the master jobset is triggered Wow, this is fantastic news. This should really help with the quality and confidence of PRs. This will hopefully also improve merge throughout. The one question I have is will this show up on the GitHub PR page or will I have to remember the URL myself. Either way it is super useful but if it could integrate with the GitHub status API that would be much easier for maintainers and new submitters alike. Huge thanks to you and Package.net. (I've been interested in running an arm server for a while so I'll check them out 😀) ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-commits] [NixOS/nixpkgs] b18657: libnl: 3.2.23 -> 2.3.29 to revert accidental downg...
Branch: refs/heads/release-16.09 Home: https://github.com/NixOS/nixpkgs Commit: b186574f257ce6c42a75e5cac366cd802ff7541e https://github.com/NixOS/nixpkgs/commit/b186574f257ce6c42a75e5cac366cd802ff7541e Author: Kevin Cox Date: 2017-01-27 (Fri, 27 Jan 2017) Changed paths: M pkgs/os-specific/linux/libnl/default.nix Log Message: --- libnl: 3.2.23 -> 2.3.29 to revert accidental downgrade. libnl was accidentally downgrades to 2.3.29 in 8d342d20b5ea7df31b0a29fc5e3664868c81d832 instead of being upgraded to 2.3.29 so this fixes that. (cherry picked from commit 6dcc4623acd4f618fa5e7e7a5fd4063c3e9d4606) ___ nix-commits mailing list nix-comm...@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-commits
[Nix-commits] [NixOS/nixpkgs] d11ed7: Properly escape systemd environment options.
Branch: refs/heads/env-quotes Home: https://github.com/NixOS/nixpkgs Commit: d11ed7b9eaf388a2d87af58b69f30a4cdcf362ae https://github.com/NixOS/nixpkgs/commit/d11ed7b9eaf388a2d87af58b69f30a4cdcf362ae Author: Kevin Cox Date: 2017-01-27 (Fri, 27 Jan 2017) Changed paths: M nixos/modules/system/boot/systemd.nix Log Message: --- Properly escape systemd environment options. Using toJSON on a string value works because the allowed JSON escape sequences is almost a subset of the systemd allowed escape sequences. The only exception is `\/` which JSON allows but systemd doesn't. Luckily this sequence isn't required and toJSON don't produce it making the result valid for systemd consumption. Examples of things that this fixes are environment variables with double quotes or newlines. ___ nix-commits mailing list nix-comm...@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-commits
Re: [Nix-dev] Is it possible to limit nix access to sudoers and/or a group?
This is a good question because installing packages can be used as a DOS attack. I wonder if it would be easy to make a patch to change the file permissions on the Nix daemon and other services that write to the Nix store. On Jan 20, 2017 02:15, "Mateusz Czaplinski" wrote: > I'd like to build a system where regular users cannot access nix > commands, daemon, etc. Ideally, only users belonging to a particular > group could access those. (Probably worse solution, but still > acceptable, if this was limited to sudoers only.) > > Is it possible? If yes, how to do that on NixOS? > > Thanks & Best Regards, > /Mateusz. > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Nix in a Cluster Environment Talk Slides
Hello, There was recently a Nix meetup in Dublin and a couple of people expressed interest in my talk on using Nix in a clustered environment. Unfortunately I fat-fingered the recording but I have posted my slides for those who find them useful. https://kevincox.ca/2017/01/19/nix-and-mesos-talk/ I'll try to make sure that we have the recording figured out for future meetups so that those who can't make it can get more value 🙂 Thanks again to Tomasz and Bariša for organizing and hosting the event, it's great so see such great attendance. Also if you find yourself in the Dublin area feel free to join us: https://www.meetup.com/NixOS-Dublin/ Cheers, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Extra dependencies for Steam games?
On 09/12/16 10:45, Nikolay Amiantov wrote: To clarify: I was talking not about Steam Runtime (the guaranteed set) but rather extra libraries that some games need (a small set defined in `multiPkgs`, under "Not formally in runtime but needed by some games" comment). Removing something from the runtime seems a bad idea to me. Oh I see. I didn't realize games did that. It seems like it would cause a lot of problems. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Extra dependencies for Steam games?
On 09/12/16 10:02, Nikolay Amiantov wrote: I'm not sure if we want to add something like `extraDependencies` argument to Steam; it feels better to me to just add those libraries unconditionally so that games work out of the box (you don't expect Steam to have low storage requirements anyway). On the other hand, there are runtimes like Mono and Java which are needed rarely and are big. Maybe we want to remove them by default and introduce the argument after all... I agree with your first point. Steam provides a guaranteed set of libraries that developers can expect to be there. I think the nixpkgs version should provide all of those libraries by default (although I am not against the possibility to remove some if you wish) ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Announcing: Security Tooling, nix-security-announce Mailing List
On Dec 3, 2016 00:01, "Graham Christensen" wrote: The list was misconfigured. We want the announce list to be announce-only and no other discussion. It is now configured to allow anyone to subscribe / join, but only certain people to send mail. For discussion about issues, I would recommend emailing nix-dev. Glad to see this fixed :) > How do I subscribe? - with a Google account, at https://groups.google.com/foru m/#!forum/nix-security-announce - without a Google account, I believe you can join the list without a Google account by emailing `nix-security-announce+subscr...@googlegroups.com `. I believe you also need the subject "subscribe" however I just used this method and it worked. Took a couple of seconds for the responses to come in but everything worked well. Thanks for setting this up Graham, it's excellent to see this security infrastructure coming into place. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Announcing: Security Tooling, nix-security-announce Mailing List
On 25/11/16 09:23, Moritz Ulrich wrote: Anders Lundstedt writes: How do I subscribe? You seem to need a Google account. But even with one, I don't see the usual subscribe button. I suppose the group is somehow misconfigured to prevent subscriptions. I'm voting to move this to a non-google-controlled server (like other Nix MLs). I'm sure there a quite a few people who don't want a Google account to stay updated on security issues. While this group is invite only you can subscribe to (public) Google Groups by email only. You simply send a message to group+subscr...@domain.tld with the subject "subscribe". You will then get an email response to which you can reply to to confirm your subscription. Once this group is made public it would be nice to add this info to the group description so that it is obviously available. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Proposal: Highly available security-specific trusted build infrastructure
On 16/10/16 18:32, Shea Levy wrote: > I think an automated system would be nicer, but yes this would resolve > the majority of my concern here. > I understand but I think Hydra works more then 95% of the time. Designing an automated system for the rare case when we need to push an emergency update sounds like it is unlikely to be fruitful. I think that if we are trying to create a backup for an automated system it would be beneficial to go quite bare bones. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Proposal: Highly available security-specific trusted build infrastructure
On 16/10/16 18:24, Shea Levy wrote: > The existing infrastructure will always have more load and be more > complex than what is needed for security updates. hydra is a fully > general CI system, and properly so, but it means the system is subject > to bugs and constraints that a simpler more focused system can avoid. > > Moreover, for better or for worse hydra.nixos.org is only manageable by > a small set of people who are not always available to service it (nor > should they have to be!). No amount of improving hydra will fix that. > I see your point. But for a emergency rebuild system for security fixes wouldn't it just make sense to have a couple of people with S3 credentials? Most packages can be built on a mildly powerful machine in an hour. In the rare case that the package would take longer it probably wouldn't be improved by a cluster as it will be a serial dependency chain. So if we really want to reduce dependencies how about a couple of people trusted to push these updated packages? signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Remove Python 2.6 and 3.3
On 26/09/16 12:29, Matthias Beyer wrote: I have 702 paths in /nix/store which contain the string "python2" Can you tell me how to check properly whether I need python2 support/packages? I'm not so familiar with the python infrastructure. I believe a lot of the "python2*" packages will be python libraries that are dependencies, so I would suspect that a vast minority would actually be the CPython interpreter. For determining what needs those paths you can use the nix-store --query (-q) to find out. I'm not on a machine right now but something like `nix-store -q --requisites` will give you everything that depends on a packages. I always get the actual wording confused but the man page has excellent descriptions in the query section. Hope that helps, Kevin ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] NixOS GitHub third party application restrictions
Hello, for contributors to the NixOS organization it is impossible to give a third party GitHub integration access to any of my organizations without also giving it access to the NixOS org. The link below has some more information about how to change this for NixOS but it will likely remove some webhooks and other auths (ex: Travis) that will need to be readded. One set third party won't get access to NixOS unless explicitly enabled. https://help.github.com/articles/about-third-party-application-restrictions/ I hope that this setting can be changed so that I can continue using third party tools for my personal uses without affecting the NixOS org. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] NixOps - How is the configuration applied when using targetEnv none?
On Sep 16, 2016 19:50, "Wilhelm Schuster" wrote: Hi, I’m trying to use NixOps to deploy onto an existing NixOs instance (physical server; targetEnv = “none") and was wondering how the system configuration is applied. Specifically, does my NixOps description of the server get merged with the local configuration (meaning the one on the server itself in /etc/nixos/configuration.nix), or does it completely replace it? It completely replaces it. In general nixops doesn't read anything from the target server. Everything is built first, then pushed to the target. The only reason that the state of the target will have any effect is if you have a runtime program that reads an explicit path. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] GitHub Octoverse 2016: nixpkgs #6 in Repositories with the most users reviewing code
On 15/09/16 13:05, Daniel Hlynskyi wrote: Who are those "reviewing users"? I didn't know about such github role. My best guess is that it is something like users who have put comments on code in repos. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] GHC "unable to decommit memory"
On Sep 1, 2016 10:03, "Eelco Dolstra" wrote: > > Hi, > > On 09/01/2016 10:09 AM, Kevin Cox wrote: > > > Sounds more like we need glibc to support the kernel we are using. GHC is > > behaving fine and probably not the only program to have this problem. > > Glibc supports our kernel just fine - it just makes available some features that > our kernel doesn't have, which is not uncommon. I mean, what should Glibc do in > this case? The fix would be to build GHC with --disable-large-address-space or > apply patch https://ghc.haskell.org/trac/ghc/ticket/12495. > It doesn't sound just fine if it is advertising features that don't exist. Is there a way to prevent glibc from saying that MADV_FREE is supported? I understand that glibc "supports" it but it probably shouldn't be advertised if the kernel underneath doesn't. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] GHC "unable to decommit memory"
On Sep 1, 2016 09:04, "Eelco Dolstra" wrote: > > > I guess we need to stop GHC from using it? We can't really have GHC not working > on our default kernel :-) > Sounds more like we need glibc to support the kernel we are using. GHC is behaving fine and probably not the only program to have this problem. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Small Question on Strings, Attributes & Functions
On 25/08/16 21:31, Vladimír Čunát wrote:
> On 08/24/2016 01:04 PM, Matan Shenhav wrote:
>> Is there a resource covering the rules for variables, values and
>> functions? I am still somewhat confused because if I define func = x:
>> x=="x" I do not need to antiquote with ${}.
>
You only need to antiquote when doing attribute access (and a few other
oddball situations). For example config.networking.hostName is is
equivalent to config."networking"."hostname". This makes attribute much
easier in the general case. So if you want to do dynamic attribute
access you need to antiquote systemd.services."${srv-name}" or if it is
solely one variable as in that example you can drop the quotes.
Note that in you example that x is always treated as a variable as it is
not in an attribute lookup. Much the same that `config` didn't need to
be quoted in the above examples.
Hope that helps, Kevin
signature.asc
Description: OpenPGP digital signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] How to install specific version of a package?
If you know the path of the package the easiest way is `nix-env -i /nix/store/...`. For installing a package by previous version number I don't think there is an easy way to do it. (Except for some packages were different versions are available in nixpkgs). This is partially because "version" in nix includes all the versions of dependencies and build instructions. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Two declarative ways to install a package?
On 11/08/16 20:11, Nick Sabalausky wrote: > > What exactly is the difference? Is there any more nuance to it than > "Services are installed one way, non-services are installed the other way"? > Basically the services are wrappers around the packages. They set them up, generate config files and set them to start running. It's also important to not that services generally (never?) actually "install" the package. Obviously it is on the system but it isn't available on your PATH. Hope that helps, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Too many open issues
On 26/07/16 16:22, Wout Mertens wrote: > Ok fair enough, so no auto-closing but a 6 month reminder would be good. > The previously mentioned bot had the option to only close when a given label was applied. I think this would be beneficial as a "need-info" label could be applied and if the OP isn't still interested it could be closed. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Requesting issue closing rights
I see granting commit access fairly scary but I would appreciate being granted it so that I can help organize the issues more effectively. I am the maintainer of a couple of packages and have a number of merged PRs so hopefully I have built up enough trust in the Nix community. https://github.com/kevincox Thanks, Kevin ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Too many open issues
On Jul 23, 2016 08:23, "obadz" wrote: > > On Sat, Jul 23, 2016 at 2:19 AM, Benno Fünfstück < benno.fuenfstu...@gmail.com> wrote: >>> >>> The key here would be that we shouldn't get rattled if we get assigned an issue/PR. All it means is "I think you know more about this than I do, feel free to pass it on to someone else if aren't the right person or can't handle this with the appropriate urgency". >> >> >> I don't think that assignment is the right tool for this job. Assignment in my opinion should be used for the purpose of avoiding duplicated work: you assign yourself to an issue if you plan on working on it, so that everyone else knows that they shouldn't work on that particular task themselves. > > > Fully agree that the purpose of assignment is to avoid duplicated work. That is why I think the first person to ever review an issue (maybe that's mentionbot) should assign someone to that issue. Once that's done, this issue won't waste anyone's else's time until the current assignee decides who would be a better assignee. This means the amount of reviewing work done is about O(n_issues). > The nice thing about assignment is it is a little harder for things to get lost. And I think asking the current assignee "I have time to take this over, is that OK?" is an awesome question to ask if it appears that nothing is being done. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Too many open issues
On 22/07/16 08:55, Alexey Shmalko wrote: > This one: https://www.codetriage.com/nixos/nixpkgs > That's it! I have subscribed to get a couple issues a day so hopefully I can help a bit. This site seems like a nice way to spread the load. It's open source too, and I just opened an issue asking them to implement filters as I'm not really interesting in seeing in-progress issues. https://github.com/codetriage/codetriage/issues/498 signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Too many open issues
I'm not a huge fan of auto-closing issues based on just activity. The one auto-close feature I do like is the `need info` tag when you need more from the reporter. Then the issue is closed in a week or two if they don't respond. The reason I don't like auto-closing issues is that it seems more like fighting the symptom then the problem. Just because you close an issue doesn't mean that it doesn't exist. I think a better way is to get more triager to quickly put issues on the wishlist or find an assignee. Rather then close it is probably better to have an "important" or tag of things that should be looked at first (and maybe try to keep that close to zero). I see little value to closing issues just because we don't want to work on them right now. It is really painful that only commiters can add tags because I would love to help out with triaging more then just commenting (I think comments often have low value because you can't filter on them). One other thing that might help is I remember a service that would send a couple of open issues to your inbox to help divide the load of triaging. I'm going to see if I can find it again as it would probably be useful. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Packaging free software that costs money
On 07/07/16 12:25, Vladimír Čunát wrote: > > As mentioned, paying for freely-licensed SW (as accepted by FSF or OSI) > is always possible but can *never* be forced (legally), so I don't think > we should make it *appear* as obligatory. > Yes, I think this is the right route. We can definitely start by defining a way to link specifically to "support" pages. I think that looking into ways to make it a little more obvious in the future might also be a good idea, such as display a message when installed or providing a command that lists all packages you have installed that request monetary support. Let's face it, most open source projects could use with more funding, and if we can help I think that is great. Of course we shouldn't make our "help" too obtrusive for the user, finding the "right" balance is always a struggle but I think that slowly working towards it is a good idea. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Packaging free software that costs money
On 06/07/16 11:17, Roger Qiu wrote: > I'm interested. Free and proprietary software should be installed in the > same way in my opinion. While that is interesting note that so far this conversation is only talking about free software. (Notably free software that requests payments for use) signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Packaging free software that costs money
On 06/07/16 09:25, Bjørn Forsman wrote: > On 6 July 2016 at 14:19, Profpatsch wrote: > > But that still looks like opt-in payment. I'm sure Ardour users > already know that upstream wants money for it, even without the above > mechanism. (But I guess if a lot of upstreams do this it'd be more > difficult to track manually, and we'd need a mechanism.) > > If upstream *really require* money, the above method seems > insufficient, IMHO. I'm starting to think about proprietary programs > now, and how you often can get the software but it doesn't work > without a license key. The key can be stored in $HOME and don't have > to have anything to do with the Nix expression that built (or just > downloaded) the software. > IIUC you can't "really require" money for a GPL program. You can charge for the distribution or for binaries or whatever you like, but nothing is stopping others from building and distributing the program for no charge (or even taking the money themselves) this is part of the freedom offered by the license. That being said I think it is nice to encourage these payments, and I think an "honour system" is enough for Nix. (again the "problem" with free software is you can only try so hard or the user will just comment out the payment check and recompile themselves) signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Nix syntax surprise.
On 24/06/16 05:58, Guillaume Maudoux (Layus) wrote:
> Hi,
>
> Does someone understand why the following nix expression is invalid ?
>
> |[ let a = 10; in { b = "a = ${a}"; } ] |
>
> Running |nix-instantiate --eval ./bug.nix --strict| gives |error: syntax
> error, unexpected ID, expecting '{', at /home/layus/bug.nix:3:5|.
>
> The manual says nothing about this in the “let expressions” section.
>
Most often if you have non-trivial expressions in a list you need to put
them in brackets. I know this doesn't specifically answer your question
but I consider it a good practice.
signature.asc
Description: OpenPGP digital signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] [y...@cock.li: Re: Malicious installation methods]
On 17/06/16 11:40, zimbatm wrote: > > I agree. For GPG to be implemented properly, the key must be distributed > separately from the content. The goal is to make the attack more > expensive by forcing the attacker to compromise multiple communication > channels. And the key fingerprint must be in the long form to mitigate > potential collision attacks. > Yes, this is the trick. Put the signing key EVERYWHERE. Sign it by the Nix maintainers and stick their keys everywhere as well. Then you verify it with as many different channels as you like. Go to a conference, read the website, different blog posts, ask a friend... Just putting the key on the website besides the installer isn't much better for the first time user, and definitely don't auto-import the key because that will remove the benefit from repeat users as well. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] [y...@cock.li: Re: Malicious installation methods]
On 17/06/16 10:33, Yui Hirasawa wrote: > >> Signing the installer script would provide only a minor increase in >> security (in that it would require the signing key to be compromised, >> rather than the nixos.org certificate). I don't object to doing that >> though. > > That is quite a major increase in security actually. Compromising a key > that can be kept offline most of the time is a lot harder than obtaining > a signed certificate for the nixos.org domain. You do not have to have > the original nixos.org certificate to perform man-in-the-middle attack. > I agree with this. A key that is trusted itself (rather then trusting a domain name) would be a very large security increase. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] [y...@cock.li: Re: Malicious installation methods]
On 17/06/16 10:26, Eelco Dolstra wrote: > > Cargo cult security is not a priority. I wouldn't worry about "curl | bash" > but > not the giant binary tarball downloaded and executed by that script (or > equivalently, installing a binary RPM or Deb package). Signing the installer > script would provide only a minor increase in security (in that it would > require > the signing key to be compromised, rather than the nixos.org certificate). I > don't object to doing that though. > I generally agree wit this. I think moving the whole system to offline signing would be nice but I don't think it's very urgent. Another advantage of moving away from the CA system is that the CA system can be bypassed if any of hundreds (thousands?) of CAs are compromised, or if the Nix servers are compromised. Where as if it is an "offline" key (even if it's an online PGP key it would be better). There is a single, more difficult attack surface. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Malicious installation methods
On 17/06/16 09:17, Adrien Devresse wrote: >> The installer, when run, will fetch more code for users to blindly execute >> (as most of that code will be provided in compiled form). How is blindly >> running an installer worse than running other code from the same provider? > > Simply put the shasum of your installer on the website and ask the user > to verify. That is what many projets do, and it's a three lines of > installation instead of one. > So you're trusting a hash from the same site that you are downloading the script from? I can see a lot of value in a cryptographic signature (like PGP) but I see almost no value in a hash. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Malicious installation methods
On 17/06/16 07:59, Azul wrote: > simple as that, > just don't do it. > > https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/ > While this is interesting research I find that it is often irrelevant because you are trusting the server anyways. So if you trust the server enough to run it's software as root you should trust it enough not to swap out the file on you. If you are paranoid curl the script and follow the steps manually. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Malicious installation methods
On 17/06/16 07:12, Yui Hirasawa wrote: > > Retrieving code straight from the internet and blindly executing is > never a good thing and you don't give any sort of recommendation for the > user to inspect the script before running it. This completely defeats > the point of having reproducible builds when your system can be > completely infected when you install the package manager. This also > means that anything installed through the package manager is potentially > malicious as well. > >> $ curl https://nixos.org/nix/install | sh > This has been discussed in many forms in many places. You are downloading code that you intend to run as root on your machine, and the distribution method is over a verified channel. This is no more dangerous then most other ways to download software that your root user will run. One improvement would be to sign the actual script with an offline key but while that would be safer the current method is perfectly fine. I know that people see `curl http...` and get all excited but, in this case at least, it is a sufficiently secure method. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] NixOps - merge config files
On 09/06/16 11:03, 4levels wrote:
> Hi Kevin,
>
> I'm very curious how you setup sensitive information using deployment.keys
> This still seems like the best option to do this, but I failed to get
> it working..
>
> Could you be so kind to post a small example, showing how you define
> the keys and use them in functions that generate eg. a config file?
Here are two examples where I read the key out of a secret directory
(encrypted with git-crypt). Note that I don't generate any of my config
files, but that is theoretically possible.
deployment.keys.mesos.text = "root ${builtins.readFile
../secret/mesos-secret}";
deployment.keys.sumologic.text = builtins.readFile ../secret/sumologic;
And this example is importing a nix expression. Note that I'm not using
deployment.keys so this is accessible to anyone (on the server) as it
lands in the Nix store IIUC.
networking.defaultMailServer = import ../secret/smtp.nix;
signature.asc
Description: OpenPGP digital signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] NixOps - merge config files
On 09/06/16 10:21, 4levels wrote: > Hi Nix Devs, > > I've tried using the deployment.keys but that seems to be only there for > user accounts and not for custom configurations. As I need multiple > keys for each platform (eg. symfony, wordpress - cipher, key, database > password, admin user password etc etc) it seems that deployment.keys > will not work for this purpose. > I don't know what you mean by "only there for user accounts". I use deployment.keys for most of the things you describe and it works fine. > I'm now resorting to using builtins.readFile and creating separate files > for the passwords in a folder I can then encrypt with git-crypt. > This way I'm also able to use conflicting nix characters in the > passwords as well. This seems like a different issue. Are you talking about keeping the keys private in the configuration or on the server (or both)? Although for keeping the keys private in the config I use git-crypt as well. I have a directory called "secrets" that I store the files in. Some I keep as text files and read using `builtins.readFile` and some I leave as nix expressions and simply import them. > > As a last question here I was wondering if there's a way to strip > trailing newlines when reading files with builtins.readFile as my IDE > keeps adding a newline at the end of each file when saving. I'll post a > new message here for this request as this is not relevant in this topic > Text files have newlines at the end 😉 You can use lib.removeSuffix to drop that newline if it's a problem. Hope that helps, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Splitting packages silently breaks a lot of stuff
On 06/06/16 06:17, Profpatsch wrote:
> In the past month or so more and more packages have been
> split into multiple outputs. I like that, but:
>
> Doing so breaks lots of dependencies silently.
>
> I think that is a very bad problem we need to address.
> Every program with runtime dependencies does build just
> fine, but breaks once its executed.
>
> Two solutions I can think of:
> 1. Programs can assert *files* they depend on directly in the
>nix expression and nix ensures these exist on build time.
> 2. Don’t use the first element in the `outputs` list by default.
>Default to a derivation of *all* outputs if no output is given
>by name.
>
> 1 might be hard to implement (afaik aszlig tried something in that
> direction) and still breaks old packages that are not updated.
> 2 will un-break all packages again and they need to be patched to
> get the closure size down again.
>
> Thoughts? Anything I’ve missed?
>
I had this same thing happen to me. I updated my NixOS and my script
that used the OpenSSL binary stopped working.
1. This would be interesting. Currently the pattern is ${pkg}/bin/exe. I
think if we could abstract that the checking could be automatic.
2. I think this train of thought more but unfortunately there is no
"all" output anymore. So you would still have to reference the right one.
Also it appears that in general the `lib` is the default output. It
seems to me like the `bin` is more useful as that is what humans would
reference more. Where as the `lib` output is used more in other build
scripts and errors would be thrown if the library isn't found.
I'm giving a definite +1 to figuring out how to reduce this breakage.
signature.asc
Description: OpenPGP digital signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Users of 32-bit NixOS channels?
Is it possible to check how many people are downloading the channel? That seems like a more reliable way to figure out who is using the channel rather then seeing who sees this in the mailing list. Cheers, Kevin. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Can a nixOps coordinator update itself?
I don't think it would be a problem but I wouldn't make promises. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Why not use SONAME instead of RPATH
On 17/04/16 12:02, Silvio Frischknecht wrote: > > Advantages over RPATH: > + probably faster since rpaths in nixos tend to be quite long and every > library has to be looked for in every folder (linear vs quadratic > complexity) > + only has to be setup once per library - all referrers will > automatically work correctly > TBH I've been wondering the same thing when reading documentation talked about rpath. This seems like the simpler more direct method. It also avoid the theoretical problem where there are two libraries of the same name in different folders, this allows you to specify exactly which one although that is unlikely to ever become a real problem. But in general in my Nix usage I try to avoid long search paths such as rpath and PATH, instead preferring to use absolute paths everywhere with no search just because it is simpler, and probably more performant. Cheers, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] runuser from utillinux not working.
Hello, I was trying to use runuser from utillinux to run a process as another user but ran into a problem. Running the command (as root) gives the following. # runuser -ukevincox whoami runuser: Failure setting user credentials # echo $? 1 A similar command using sudo works but I was wondering why runuser doesn't. Looking around on the web I found that this error could be caused by missing PAM config files[1,2] however I don't really know PAM at all and was wondering if someone knew what the problem was in our case. [1] http://patchwork.openembedded.org/patch/104339/ [2] https://bugs.mageia.org/show_bug.cgi?id=6391 Thanks, Kevin signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] nix-shell is unable to install the package which nix-env just returned
On 30/03/16 06:40 PM, Dmitry Malikov wrote: > nix-shell -p letsencrypt nix-shell -p pkgs.letsencrypt Cheers signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Did we just get windows support for free?
Hello Nixers, Microsoft has announced that Windows 10 will have a Linux ABI. What this means is that it will be able to run native Linux binaries by presenting them with a Linux compatible system call table. Since the only dependency Nix has on the underlying OS is the kernel ABI I believe that this means that you should be able to use Nix on Windows 10 just by treading it as x86 Linux. I doubt GUI apps will work because of the different display server and whatnot but I imagine that almost all apps will work without any changes or recompiling. Very slow link: http://www.hanselman.com/blog/DevelopersCanRunBashShellAndUsermodeUbuntuLinuxBinariesOnWindows10.aspx Google cache: https://webcache.googleusercontent.com/search?q=cache:http://www.hanselman.com/blog/DevelopersCanRunBashShellAndUsermodeUbuntuLinuxBinariesOnWindows10.aspx More mediaish link: http://blog.dustinkirkland.com/2016/03/ubuntu-on-windows.html I'm interesting in hearing what you guys think? signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] When calling nix-store --verify-path - How to know the hash database is not corrupt?
On Mar 10, 2016 5:28 AM, "Eelco Dolstra" wrote: > > > In the future Nix will probably store binary cache signatures in its database, > and provide a command to check local paths against binary caches. > The problem with this is that if you are running a local command to validate signatures you need to trust the local command which you can't. I'm not saying that it is useful to provide these tools but if you are paranoid enough the only safe thing to do once you suspect someone has comprised your box is to burn it. Another alternative which runs the risk of not checking that firmware hasn't been modified is mounting the disk in a trusted system and running the check from there. Pick your desired security level :) ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] AppImage
Well the key differences I see are: Nix: - Dependencies are shared, less disk space and better cache usage. - Requires software on the system to manage the programs. AppImage: - Dependencies are bundled. - Everything comes in a "standard" ELF executable so it can run anywhere (provided a compatible kernel and that you have linked in enough libs) So while I think it is cool that it is simply a binary you can run you obviously lose out on the efficiency of Nix. signature.asc Description: OpenPGP digital signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Printing test failure preventing unstable channel update
The raw log links were working for me. I don't know why the others aren't. On Feb 20, 2016 13:08, "Jascha Geerds" wrote: > Unfortunately, I don't see any logs for the build > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Systemd User Timers: timerConfig section?
I don't know if it is missing for user timer units but there is a
timerConfig. However defining repeating timers in systemd is weird so I
have included an example that I use below.
systemd.timers.sysstat-journald = {
wantedBy = [ "multi-user.target" ];
timerConfig = {
OnBootSec = "1m";
OnUnitActiveSec = "20s";
AccuracySec = "1s";
};
};
The key elements are "OnBootSec" which is when to run the first timer
after boot, and "OnUnitActiveSec" which is how long to wait before
running it again.
Hope that helps.##SELECTION_END##
signature.asc
Description: This is a digitally signed message part
smime.p7s
Description: S/MIME cryptographic signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Debugging Symbols
On Mon, 2016-02-15 at 17:50 +0100, Eelco Dolstra wrote: > This is already supported. Just add > separateDebugInfo = true; > > to a package to get an output named "debug", with debug symbols in > lib/debug/.build-id. This also enables the -ggdb flag. > > You can then enable GDB to find these symbols by adding > > environment.enableDebugInfo = true; > > to configuration.nix. > > Note however that very few packages in Nixpkgs currently set separateDebugInfo > because Hydra disk space is problematic as it is. > > That sounds fantastic. I'll try it out. I didn't realize that the enableDebugInfo flag was required. On a somewhat related note, I was wondering why the environment links specific directories, rather then automatically linking everything? signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Debugging Symbols
On Mon, 2016-02-15 at 17:45 +0100, Eelco Dolstra wrote: > Hi, > > On 15/02/16 15:43, Kevin Cox wrote: > > > The solution is obviously seperate debug files > > > > l> which > > allows the debugging symbols to be kept separate from the binary > > itself. I see > > that there was an earlier attempt but it appears to have failed due > > to profile > > linking issues > > (https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/config/ > > debug-info.nix). > > Not sure I understand the problem. That *is* the supported way on > NixOS to get > debug symbols, and AFAIK it works. What's the profile linking issue > you refer to? Sorry, I guess I was misunderstanding the comments. I interpreted it as it was disabled due to not being able to create a symlink forest for /lib/debug/.build-id when /lib is enabled. But reading the file again it looks like this is a work-around for that issue? > > > > I also think that this approach isn't ideal because it requires the program > > to > > be "installed" which is generally not the case for my system services. > > > > > Yes, currently it only includes debug info for packages in > environment.systemPackages, but this could fairly easily be extended to > packages > in the path of systemd services, or maybe even all packages in the system > closure. You can also explicitly include debug symbols, e.g. > > environment.systemPackages = [ hello.debug ]; > Ok, if this is working it isn't a bad solution. Is there any way to automatically link all of the dependant packages into the environment? Because often you will want things like libc and other used libraries as well. Or is this currently a manual process. > (At FOSDEM, Mark Wielaard [1] mentioned that we should really have a little > FUSE > filesystem to make a repository of *all* debug symbols available to GDB over > the > network, so you don't have to install them manually. That would be really > neat...) > > [1] > https://fosdem.org/2016/schedule/event/where_are_your_symbols_debuginfo_and_sources/ That would be really cool but considerably more work. We would also have to figure out how to map a build-id to a store path. An external index wouldn't be super pretty but it would work. Sorry for the noise then, I didn't realize that this was working. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Debugging Symbols
On Feb 15, 2016 10:13 AM, "Adrien Devresse" wrote: > > +1 to this proposal. Thanks > > - It happens often that recompiling with debug flags change the behavior of the progrm itself and make the crash/problem impossible to reproduce with debinfos. > I would like to point out that this is only about compiling with debug symbols, not about changing the optimization level. GCC and clang support this although apparently some compilers don't. The executable code should be identical. > - It is a pain when a crash happen in a low level library ( e.g libc ) and that half of the stack has to be recompiled just to get some debug informations. > This is a great point. There should probably be an easy way to get all of the required symbols because downloading one by one will be a pain. This should be fairly easy to do with a python gdb extension. > Having an additional output with the deb symbols per derivation would be similar to the others packaging system (RPM, Deb) approach ( *-dbg.deb, *-debuginfo.rpm ). > Definitely inspired. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Debugging Symbols
On Mon, 2016-02-15 at 15:50 +0100, Luca Bruno wrote: > > > +1, with the multi outputs branch [1] it makes a lot of sense to ship > an additional output for debug symbols, instead of rebuilding the > program using Nix functions [2] > > [1] https://github.com/NixOS/nixpkgs/pull/7701 > [2] https://github.com/NixOS/nixpkgs/blob/master/pkgs/misc/misc.nix#L > 113 Definitely! Having the debug function would still be useful because debugging optimized programs is often difficult but the symbols version can go away because all programs would have symbols already. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Debugging Symbols
Hello all, I'm creating a bit of noise this morning.
I noticed that almost all Nix packages are stripped of debugging
symbols and was hoping to discuss a possible solution that would both
keep the closure size minimal and allow post-hoc debugging of crashes.
The small closure size speaks for itself and since systemd helpfully
stores coredumps of all crashes it would be nice to be able to look
into them without having to recompile and observe the crash again.
The solution is obviously seperate debug files which allows the
debugging symbols to be kept separate from the binary itself. I see
that there was an earlier attempt but it appears to have failed due to
profile linking issues (https://github.com/NixOS/nixpkgs/blob/master/ni
xos/modules/config/debug-info.nix). I also think that this approach
isn't ideal because it requires the program to be "installed" which is
generally not the case for my system services.
Here is my proposal and please feel free to critique it.
- Change the strip helper to copy the debugging info first (and start
building with max debug info (-ggdb)).
- Store the debugging info in it's output ("symbols" or "debug") under
the same file path of the original binary.
- Embed the path of the debug symbols in the .gnu_debuglink section
using a hacky path.
The assumptions I am under are that there can only be one debug output
(because nix isn't dynamic enough for determining outputs at build
time) and that directories with paths continue to work in gdb.
A bit of background is that the .gnu_debuglink section is intended to
contain the basename of the debugging symbols which is looked up in the
global debug directories. However experimentation shows that while the
tools make it difficult to create a .gnu_debuglink with slashes in it
gdb will faithfully concatenate the paths and try to read the symbols
there. Using this we can set the global debug directory to /nix/store,
effectively allowing us to specify any path as the debug symbols.
That means for a file /nix/store/xxx-foo/bin/exe the build scripts
should copy the debugging symbols over to /nix/store/yyy-foo-
debug/bin/exe.debug and set the debug link to yyy-foo-
debug/bin/exe.debug. Then whenever gdb is started with exe or a
crashdump of exe it will automatically pick up the debugging symbols
(assuming they are installed).
The one downside to this approach is that installing the debugging
symbols is non-obvious. However it would be trivial to create a simple
script that takes an executable as an argument and reads the
.gnu_debuglink section out of it then realizes the path. Assuming that
you have that path in a binary cache or a store derivation it should
create those files without issue.
Another approach that I tested was to create a small python extension
(a couple of lines, below for reference), either embedded in the binary
or alongside the executable on the filesystem, that tried to load the
debug symbols by path, then if it failed printed a message with the
realization command. That way the use could just copy the command to
get the debugging symbols. However this wasn't ideal as it took more
space in the executable (roughly 6x, which really isn't much) and
required including the whole store in the "auto-load safe-path" in
order to have a seamless experience. Of course the advantage to this is
if you don't have auto-load enabled for a particular file you get a
nice message from gdb telling you how to enable it.
f=" /nix/store/yyy-foo-debug/bin/exe"
try:gdb.execute("symbol-file"+f);print("Debugging symbols loaded")
except:print('Error loading debugging symbols. They can be downloaded
with "nix-store -r"'+f);raise
I'm also going to look into other gdb extension options to do this from
a trusted extension with less size required in the actual packages.
I'll post to the list if I find something promising.
TL;DR debugging symbols for all binaries, can be downloaded after the
fact, works automatically in gdb.
signature.asc
Description: This is a digitally signed message part
smime.p7s
Description: S/MIME cryptographic signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Travis Testing Needs Rethinking
On Sat, 2016-02-13 at 17:03 +, Adam Russell wrote: > Wouldn't this all be less of an issue if the build on Hydra wasn't > behind by weeks? Should we talk about how to improve that? Personally > I don't even know how to navigate or interpret Hydra when I go look > at it. > I don't think Hydra is actually behind much, the channels are but IIUC most builds are actually completing in a couple of hours. (Actually looking now I see some 2 day old builds but nothing too terrible http:/ /hydra.nixos.org/status). Getting faster hydra builds might make this less of an issue but there will always be a delay. Of course if that delay is very small it won't matter much. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Travis Testing Needs Rethinking
On Sat, 2016-02-13 at 17:48 +0100, Jakob Gillich wrote: > I assume PR branch refers to unstable? Or even a stable branch? I > don't really see how testing against non-master is better when you're > submitting something for master. The PR will still be based of master but master as of a couple of hours/days ago. That way master has had a little bit of time to settle. The problem is that once you merge it in you get the very latest master which often has a number of non-uploaded packages. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Travis Testing Needs Rethinking
Hello, I have been trying to submit a pull request for quite some time but keep getting bitten by the Travis testing. The problem boils down to a couple of things. Firstly travis has build limits both in terms of log size and build time. These are expected and while there is possibility of getting them raised (at least for log size) I'm going to assume that these are the bounds we have to work within. The reason that this is a problem is because of the way nox (the review tool) works. Nox merges your new changes into the master branch then attempts to build your PR. The problem with this is that anything you depend on that has recently been changed will have to be rebuilt from source as they aren't yet in a binary cache. When you have a sufficiently complex package you almost always have dependencies that have recently changed so you end up rebuilding a lot of unchanged packages which will often send you over your resource quotas. This is especially true as these complex packages are often close to the limits on their own. To combat this problem I propose that packages are tested on their current branch. To determine what has changed simple identify the changes since the latest common ancestor of the pull request and the master branch. This means that you can allow your PR to get slightly behind master and now all of your dependencies are in the binary cache and don't eat up your resources. The apparent downside is that you aren't testing on the latest code so there may have been incompatible changes in the mean time that appear on merge. However since there is often a couple of days delay between Travis passing and merging of PRs anyways so I don't see this as much added risk. If desired an error or warning can be added when a PR is "too far" behind master to limit this problem as well. In the rare case of conflict it will be caught on the master branch, either by testing master in a similar way (although finding the "base" commit might be non-trivial and you still have the resource usage issue) or when they appear on hydra. Thanks to git and Github reverting PRs that cause breakage is literally two clicks so anything that causes a merge conflict can be reverted and the submitter notified so that it can be fixed up to be re-merged. I think that the overall value proposition is a benefit as it will make merging PRs much less painful because build times will be down which prevents travis killing builds and enables faster feedback and iteration. TL;DR building PRs based off of latest master often takes too many resources for Travis so we should base CI builds off of the PR branch. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Including binary-only derivation in nixops deploy.
On Feb 12, 2016 6:54 AM, "Oliver Charles" wrote:
>
> If you're saying you want to install something knowing only the Nix store
path, you can do:
>
> { type = "derivation";
> name = "whatever";
> outputName = "out";
> outPath = builtins.storePath "/nix/store/deadbeefdeadbeef"; }
>
> This will attempt to lookup the given store path in binary substitution
servers, and will fail if it can't be found (as Nix has no other way to
provide that output).
>
> Does that help?
>
Thanks Oliver, that's exactly what I was looking for.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Including binary-only derivation in nixops deploy.
Hello all. I have a possibly strange question but I was hoping that you could help me out. I am currently using nixops to manage a couple of servers and have a couple custom derivations for personal packages. While managing the derivation in my nixops config isn't terrible I wanted to manage the derivations in each project so that I can use nix to build the project in CI and use nix-shell for development. Another benefit of using a binary is that I know that dependencies won't change, and that the package has been tested with the exact dependencies it will be shipping to production. So long story short I have uploaded the derivation to a binary cache and I was hoping to use it in a nixops deploy. However I can't figure out how to get it to work (if it is possible). `nix-store -r /nix/store/01kymbkb0a1sc99y72wzn5b4gcjdlbnm-dontsayit-nginx` works fine, the package and all of its dependencies are installed to the local store. However I can't figure out how to use it in a nixops config. The naive approach of just including the path in a file (in my case an nginx config) doesn't work as it isn't being scanned for that path and the dependency doesn't get picked up. Interpolating it as a path also doesn't work as the file gets copied from the store and put into the store under a different hash, losing all dependency information. I tried playing around with .drv descriptions but couldn't figure out how to get it to work either. I would appreciate any help or recommendations for alternative solutions. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Notes and ideas about the Nix-UI proposal
On Sat, 2016-01-23 at 17:59 +0100, Matthias Beyer wrote: > > As a suggestion to the transactional behaviour, I’d like to avoid > > unnecessary shell quoting. How about “+” instead of “;"? I could > > imagine another non-colliding operator from the nix language, but > > “;” as a statement separator is going to be confusing because bash > > already uses it for almost the same thing. > > > > I want to second this! Especially because "\" is kinda hard to type > on german > keyboards (two strokes and a weird location of both keys, > ergonomically). "+" > sounds much better. > > What would be nice is a "repl"-like thing where you can do this: > > nix transaction > > install firefox > > uninstall chromium > > commit ## or maybe ctrl-D > [nix] transaction about to be committed... doing things now > > (ofc, the `nix install firefox $sep uninstall chromium` variant > should be > available as well) I think it might be even nicer to just allow multiple packages in install without having to specify the flag again. You can then have a special prefix to uninstall. It is a little "stranger" but I doubt that many new users are worried about grouping things in transactions. I'm imagining something like: nix install firefox gimp ~chromium # Install Firefox and gimp, remove chromium It could also be supported for uninstall to be symmetrical if desired. I don't know if I'm too sold on this one. nix uninstall blender openttd ~thunderbird # Remove first two install the second. The downside is that this is limited to install and uninstall but I don't know if there is anything else that would need to be grouped into a transaction. signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Perl -> C++
On Dec 30, 2015 21:21, "Arseniy Seroka" wrote: > > In **my opinion** go is not a good language. And lua is not a language that is needed in that kind of program. > I have to agree with this. Rust is awesome for it's strong type system but c++ is so standard that it might be worth sticking with it. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Importing .narinfo into nix store.
On Mon, 2015-12-21 at 14:25 +0100, Eelco Dolstra wrote: > > Yes, "nix-store -r $storepath", e.g. > > $ nix-store -r /nix/store/ln4j3vq4vkdh1fldjwqwjh49cl4l0q6k-hello- > 2.10 \ > --option binary-caches http://my-cache.example.org > > (This assumes that the calling user has sufficient privileges to > specify > binary caches.) > > Under the hood this runs the download-from-binary-cache substituter > that > Vladimir mentioned. > Thanks, those are exactly the tools I was looking for. I didn't get that you could realize from a cache. Cheers, Kevin signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Importing .narinfo into nix store.
Hello everyone, I have been trying to get a cluster of computers set up running Nix and it has been working really well so far. However I am trying to do something possible a little strange and was looking for a little help or suggestions for alternative methods of accomplishing my goal. I am trying to get a method for downloading the dependencies of a program and run it. This is similar to `nix-shell` but I want to use precompiled binaries exclusively instead of downloading a Nix expression and evaluating it. I have most of the pieces in place, I can build my program and use nix- push to generate a binary cache of the program as well as all of its dependencies. This basically leaves me with a file called hash.narinfo in my binary cache. I feel like it should be fairly easy to use that file to pull down all of the dependencies required into the nix store so that my program can run, however I can't find a way to do this. `nix-store --import` seems very close but it works on a different input format that includes all dependencies into a single file. Is there a similar command that can work with binary caches over the web? I hope my rambling made sense, and thanks for any suggestions. signature.asc Description: This is a digitally signed message part ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
