Re: [Nix-dev] Openssl and fast security updates
Thanks Vlad, just wondering I don't see Gnu TLS or PolarSSL as options already built. I can only see OpenSSL as an already built package, so I've got to create a nix package first? Aloha, RK. On Wed, Jun 11, 2014 at 4:12 PM, Vladimír Čunát wrote: > On 06/11/2014 08:07 AM, Raahul Kumar wrote: > >> How do I get Nixos to link with one of the other SSL library alternatives. >> > > I'd use standard packageOverrides to redefine openssl attribute. > > Vlada > > > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 06/11/2014 08:07 AM, Raahul Kumar wrote: How do I get Nixos to link with one of the other SSL library alternatives. I'd use standard packageOverrides to redefine openssl attribute. Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Just wondering incidentally, similarly to how Fedora has switched to the Mozilla library, Debian uses GNUTLS, maybe it's time to switch to another SSL library. How do I get Nixos to link with one of the other SSL library alternatives. I think PolarSSL looks the best of the bunch. But any alternative sounds good at this point. On Tue, Jun 10, 2014 at 2:07 AM, Luca Bruno wrote: > In the meanwhile I've merged system.replaceRuntimeDependencies. Here's > some docs: https://nixos.org/wiki/Security_Updates > > > On Fri, Jun 6, 2014 at 3:20 PM, Eelco Dolstra > wrote: > >> Hi, >> >> On 06/06/14 13:57, Mathijs Kwik wrote: >> >> > So server systems need to start following a channel that does not run >> > lengthy tests? >> >> The NixOS system tests actually aren't very lengthy (though they do >> suffer from >> non-deterministic failures). It's more the latency of having to wait for >> 20,000 >> package builds. >> >> Obviously, a better solution would be to: >> >> * Throw more hardware at the problem. >> >> * Start mirroring NARs to cache.nixos.org before the jobset eval has >> finished >> building. >> >> -- >> Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ >> ___ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev >> > > > > -- > www.debian.org - The Universal Operating System > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
In the meanwhile I've merged system.replaceRuntimeDependencies. Here's some docs: https://nixos.org/wiki/Security_Updates On Fri, Jun 6, 2014 at 3:20 PM, Eelco Dolstra wrote: > Hi, > > On 06/06/14 13:57, Mathijs Kwik wrote: > > > So server systems need to start following a channel that does not run > > lengthy tests? > > The NixOS system tests actually aren't very lengthy (though they do suffer > from > non-deterministic failures). It's more the latency of having to wait for > 20,000 > package builds. > > Obviously, a better solution would be to: > > * Throw more hardware at the problem. > > * Start mirroring NARs to cache.nixos.org before the jobset eval has > finished > building. > > -- > Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > -- www.debian.org - The Universal Operating System ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Hi, On 06/06/14 13:57, Mathijs Kwik wrote: > So server systems need to start following a channel that does not run > lengthy tests? The NixOS system tests actually aren't very lengthy (though they do suffer from non-deterministic failures). It's more the latency of having to wait for 20,000 package builds. Obviously, a better solution would be to: * Throw more hardware at the problem. * Start mirroring NARs to cache.nixos.org before the jobset eval has finished building. -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 06/06/2014 07:59 AM, Ertugrul Söylemez wrote: > On Thu, 05 Jun 2014 23:39:34 +0200 > Vladimír Čunát wrote: > >> Hydra has and uses priorities. Anyway, building OpenSSL itself is very >> quick, but rebuilding all that (transitively) depends on it is worse. >> And there are CVE fixes for stdenv stuff sometimes (glibc)... > Yes, and the basic idea is that you could have high priority packages like > OpenSSL, OpenVPN and nginx. Whenever Hydra sees a job of higher priority it > starts doing it (potentially aborting whatever it is currently doing). Once > all jobs of the same priority are done, it runs the tests of the same > priority and updates the channel. Then it goes to the next highest priority. > That way security updates won't take longer than necessary. > > When we use priorities generously we could avoid a lot of delay even in less > critical cases. > > > Greets, > Ertugrul > So you would provide a separate channel then which is updated incrementally, and the original channel is updated once all packages are being built? Does the nix package manager allow to measure how many packages it would have to build for a system's current configuration and for how many packages there would be binary package available? If so, it would help to decide whether to wait for Hydra to build the needed packages, or when to start building the remaining locally. Best, Thomas ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Eelco Dolstra writes: > Hi, > > On 06/06/14 13:29, Mathijs Kwik wrote: > >> How are people supposed to use that channel? >> I don't think I can _add_ a secondary channel which provides a >> conflicting source (nixos). Switching back and forth doesn't sound >> easy. > > Switching to another channel is very easy: > > $ nix-channel --add http://nixos.org/channels/ nixos The command is not the uneasy part :) I just don't think that switching back&forth for security-updates is a sane thing to do. Leaving the system at the "security" channel means you will end up with untested stuff that probably fails the default channel build. I would rather have a secondary channel that gets "merged" (packageOverrides style) into nixpkgs, which only contains overrides that haven't made it into the main channel yet. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Hi, On 06/06/14 13:14, Mathijs Kwik wrote: > That still doesn't solve the question "how to do a half build". > If there have been 5 commits since the last channel build, and you add > an important security fix, there is no way to only build that fix > without building the other stuff. Well, if it's really urgent, you can always revert those 5 commits :-) -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Michael Raskin <7c6f4...@mail.ru> writes: Calculating the transitive closure for all nixos modules / services run by systemd is one way to prioritize. A populatiry contest could be added to that. >>> >>> Maybe having a channel which is a subset of the main channel and >>> includes at least ssh, apache, nginx, postgresql, mysql, and some ftp >>> server would be a nice start? >> >>How are people supposed to use that channel? >>I don't think I can _add_ a secondary channel which provides a >>conflicting source (nixos). Switching back and forth doesn't sound >>easy. >> >>Also, this would just make your system start to build all the additional >>packages (not built by this new channel) by itself. No matter how much >>stuff gets pre-built, a channel always contains a specific nixpkgs >>version, so with or without binary archives, a nixos-rebuild _will_ >>build everything for that release. > > This will be a channel for server systems. You will checkout master and > do a rebuild, and this channel would provide you with prebuilt packages > for most of your server needs. So server systems need to start following a channel that does not run lengthy tests? And I think people don't usually use local git repos on their servers. The nixpkgs/nixos tree gets provided by the channel. I just want to run `nix-channel --update && nixos-rebuild switch` on my server. Not supply it with a local git checkout that I need to keep up to date and do custom rebuilds against. I do see what you are aiming for, but I don't think there's a clean way to combine it with the default workflow (nicely tested channel which provides nixpkgs/nixos sources and binaries). If there's a way to be able to add _both_ channels, this would be great. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Hi, On 06/06/14 13:29, Mathijs Kwik wrote: > How are people supposed to use that channel? > I don't think I can _add_ a secondary channel which provides a > conflicting source (nixos). Switching back and forth doesn't sound > easy. Switching to another channel is very easy: $ nix-channel --add http://nixos.org/channels/ nixos -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
>>>Calculating the transitive closure for all nixos modules / services run by >>>systemd is one way to prioritize. A populatiry contest could be added to >>>that. >> >> Maybe having a channel which is a subset of the main channel and >> includes at least ssh, apache, nginx, postgresql, mysql, and some ftp >> server would be a nice start? > >How are people supposed to use that channel? >I don't think I can _add_ a secondary channel which provides a >conflicting source (nixos). Switching back and forth doesn't sound >easy. > >Also, this would just make your system start to build all the additional >packages (not built by this new channel) by itself. No matter how much >stuff gets pre-built, a channel always contains a specific nixpkgs >version, so with or without binary archives, a nixos-rebuild _will_ >build everything for that release. This will be a channel for server systems. You will checkout master and do a rebuild, and this channel would provide you with prebuilt packages for most of your server needs. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Michael Raskin <7c6f4...@mail.ru> writes: When we use priorities generously we could avoid a lot of delay even in less critical cases. >>> >>> The main problem I see is that normally you don't want to release a >>> channel until *all* parts have rebuilt. >>> >> >>+1 Rebuilding for a server that runs, say ssh, apache, nginx, postfix and a >>few such services takes maybe 2% of the time required to build a full >>desktop distribution. >> >>I think being able to release packages used on public facing servers could >>be prioritized over, say LibreOffice, Qt, Webkit etc. >> >>If the system environment is not "polluted" by the desktop packages, it >>could be possible to upgrade the system environment before user >>environments that needs one or two orders of magnitude more time to compile. >> >>Calculating the transitive closure for all nixos modules / services run by >>systemd is one way to prioritize. A populatiry contest could be added to >>that. > > Maybe having a channel which is a subset of the main channel and > includes at least ssh, apache, nginx, postgresql, mysql, and some ftp > server would be a nice start? How are people supposed to use that channel? I don't think I can _add_ a secondary channel which provides a conflicting source (nixos). Switching back and forth doesn't sound easy. Also, this would just make your system start to build all the additional packages (not built by this new channel) by itself. No matter how much stuff gets pre-built, a channel always contains a specific nixpkgs version, so with or without binary archives, a nixos-rebuild _will_ build everything for that release. > > > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Hi, On 06/06/14 13:13, Michael Raskin wrote: > Maybe having a channel which is a subset of the main channel and > includes at least ssh, apache, nginx, postgresql, mysql, and some ftp > server would be a nice start? Yeah, that sounds like a good idea. -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Alexander Kjeldaas writes: > On Fri, Jun 6, 2014 at 10:20 AM, Vladimír Čunát wrote: > >> On 06/06/2014 08:59 AM, Ertugrul Söylemez wrote: >> >>> When we use priorities generously we could avoid a lot of delay even in >>> less critical cases. >>> >> >> The main problem I see is that normally you don't want to release a >> channel until *all* parts have rebuilt. >> > > +1 Rebuilding for a server that runs, say ssh, apache, nginx, postfix and a > few such services takes maybe 2% of the time required to build a full > desktop distribution. > > I think being able to release packages used on public facing servers could > be prioritized over, say LibreOffice, Qt, Webkit etc. > > If the system environment is not "polluted" by the desktop packages, it > could be possible to upgrade the system environment before user > environments that needs one or two orders of magnitude more time to compile. > > Calculating the transitive closure for all nixos modules / services run by > systemd is one way to prioritize. A populatiry contest could be added to > that. That still doesn't solve the question "how to do a half build". If there have been 5 commits since the last channel build, and you add an important security fix, there is no way to only build that fix without building the other stuff. A build takes a full nixpkgs checkout and builds that. In its output, it includes a tarball with the nixpkgs tree that was used. The only way to _only_ get the fix is to have a separate git branch (on top of what? release-14.04?) with just these changes and a separate hydra target/job that only builds a few important server packages and does not perform the expensive run-in-vm tests. But even then, I don't know how this would magically end up on people's systems, because I don't think you can "merge 2 channels" that both provide nixpkgs/nixos. > > Alexander > > >> >> We do have meta.schedulingPriority, but it's used little, and from earlier >> discussions I think it's really hard to objectively determine which >> packages are more important than others ;-) >> >> BTW, aborting jobs would need to be done very carefully, because we have >> some jobs that run for hours, so that could lead to wasting lots of time. >> >> >> Vlada >> >> >> >> ___ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev >> >> > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
>>> When we use priorities generously we could avoid a lot of delay even in >>> less critical cases. >>> >> >> The main problem I see is that normally you don't want to release a >> channel until *all* parts have rebuilt. >> > >+1 Rebuilding for a server that runs, say ssh, apache, nginx, postfix and a >few such services takes maybe 2% of the time required to build a full >desktop distribution. > >I think being able to release packages used on public facing servers could >be prioritized over, say LibreOffice, Qt, Webkit etc. > >If the system environment is not "polluted" by the desktop packages, it >could be possible to upgrade the system environment before user >environments that needs one or two orders of magnitude more time to compile. > >Calculating the transitive closure for all nixos modules / services run by >systemd is one way to prioritize. A populatiry contest could be added to >that. Maybe having a channel which is a subset of the main channel and includes at least ssh, apache, nginx, postgresql, mysql, and some ftp server would be a nice start? ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On Fri, Jun 6, 2014 at 10:20 AM, Vladimír Čunát wrote: > On 06/06/2014 08:59 AM, Ertugrul Söylemez wrote: > >> When we use priorities generously we could avoid a lot of delay even in >> less critical cases. >> > > The main problem I see is that normally you don't want to release a > channel until *all* parts have rebuilt. > +1 Rebuilding for a server that runs, say ssh, apache, nginx, postfix and a few such services takes maybe 2% of the time required to build a full desktop distribution. I think being able to release packages used on public facing servers could be prioritized over, say LibreOffice, Qt, Webkit etc. If the system environment is not "polluted" by the desktop packages, it could be possible to upgrade the system environment before user environments that needs one or two orders of magnitude more time to compile. Calculating the transitive closure for all nixos modules / services run by systemd is one way to prioritize. A populatiry contest could be added to that. Alexander > > We do have meta.schedulingPriority, but it's used little, and from earlier > discussions I think it's really hard to objectively determine which > packages are more important than others ;-) > > BTW, aborting jobs would need to be done very carefully, because we have > some jobs that run for hours, so that could lead to wasting lots of time. > > > Vlada > > > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 06/06/2014 08:59 AM, Ertugrul Söylemez wrote: When we use priorities generously we could avoid a lot of delay even in less critical cases. The main problem I see is that normally you don't want to release a channel until *all* parts have rebuilt. We do have meta.schedulingPriority, but it's used little, and from earlier discussions I think it's really hard to objectively determine which packages are more important than others ;-) BTW, aborting jobs would need to be done very carefully, because we have some jobs that run for hours, so that could lead to wasting lots of time. Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On Thu, 05 Jun 2014 23:39:34 +0200 Vladimír Čunát wrote: > Hydra has and uses priorities. Anyway, building OpenSSL itself is very > quick, but rebuilding all that (transitively) depends on it is worse. > And there are CVE fixes for stdenv stuff sometimes (glibc)... Yes, and the basic idea is that you could have high priority packages like OpenSSL, OpenVPN and nginx. Whenever Hydra sees a job of higher priority it starts doing it (potentially aborting whatever it is currently doing). Once all jobs of the same priority are done, it runs the tests of the same priority and updates the channel. Then it goes to the next highest priority. That way security updates won't take longer than necessary. When we use priorities generously we could avoid a lot of delay even in less critical cases. Greets, Ertugrul -- Ertugrul Söylemez ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
I believe that is a separate delay. Both exist Am 06.06.2014 06:49 schrieb "Michael Raskin" <7c6f4...@mail.ru>: > >Note that we're currently not just waiting for Hydra, but also for the > >delayed appearance on the official cache.nixos.org, which AFAIK can > >take something like a day. > > As far as I understand, this delay is the delay of Hydra building the > entire channel. I.e. fresh Nginx will not go to the cache until > LibreOffice in the same channel is also rebuilt. > > >2014-06-05 22:50 GMT+02:00 Ertugrul Söylemez : > >> On Thu, 5 Jun 2014 21:01:59 +0100 > >> Shell Turner wrote: > >> > >>> So is the argument that it should be possible to update the channel > >>> with the new package definition before the binary cache has finished > >>> building, thus letting people rebuild their systems locally if need > >>> be? That seems reasonable. > >> > >> I think a nice solution would be to add build priorities to Hydra. > When a security update is required quickly, then update the OpenSSL > expression, assign a high build priority to OpenSSL and the common server > packages and let Hydra do the building. Most people will build on weaker > machines, so I think that "waiting for Hydra" is the way to go, even when > you can't watch the actual build process and thus feel that nothing is > happening. > >> > >> All we need is to make sure that Hydra builds those quickly enough. > > > > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
>Note that we're currently not just waiting for Hydra, but also for the >delayed appearance on the official cache.nixos.org, which AFAIK can >take something like a day. As far as I understand, this delay is the delay of Hydra building the entire channel. I.e. fresh Nginx will not go to the cache until LibreOffice in the same channel is also rebuilt. >2014-06-05 22:50 GMT+02:00 Ertugrul Söylemez : >> On Thu, 5 Jun 2014 21:01:59 +0100 >> Shell Turner wrote: >> >>> So is the argument that it should be possible to update the channel >>> with the new package definition before the binary cache has finished >>> building, thus letting people rebuild their systems locally if need >>> be? That seems reasonable. >> >> I think a nice solution would be to add build priorities to Hydra. When a >> security update is required quickly, then update the OpenSSL expression, >> assign a high build priority to OpenSSL and the common server packages and >> let Hydra do the building. Most people will build on weaker machines, so I >> think that "waiting for Hydra" is the way to go, even when you can't watch >> the actual build process and thus feel that nothing is happening. >> >> All we need is to make sure that Hydra builds those quickly enough. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 06/05/2014 10:50 PM, Ertugrul Söylemez wrote: I think a nice solution would be to add build priorities to Hydra. When a security update is required quickly, then update the OpenSSL expression, assign a high build priority to OpenSSL and the common server packages and let Hydra do the building. Most people will build on weaker machines, so I think that "waiting for Hydra" is the way to go, even when you can't watch the actual build process and thus feel that nothing is happening. Hydra has and uses priorities. Anyway, building OpenSSL itself is very quick, but rebuilding all that (transitively) depends on it is worse. And there are CVE fixes for stdenv stuff sometimes (glibc)... Also, as noted, channel will NOT update until all packages are finished and tests succeed. For a big rebuild that takes days. Some non-deterministic test failures can delay it, too. Vlada smime.p7s Description: S/MIME Cryptographic Signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Note that we're currently not just waiting for Hydra, but also for the delayed appearance on the official cache.nixos.org, which AFAIK can take something like a day. 2014-06-05 22:50 GMT+02:00 Ertugrul Söylemez : > On Thu, 5 Jun 2014 21:01:59 +0100 > Shell Turner wrote: > >> So is the argument that it should be possible to update the channel >> with the new package definition before the binary cache has finished >> building, thus letting people rebuild their systems locally if need >> be? That seems reasonable. > > I think a nice solution would be to add build priorities to Hydra. When a > security update is required quickly, then update the OpenSSL expression, > assign a high build priority to OpenSSL and the common server packages and > let Hydra do the building. Most people will build on weaker machines, so I > think that "waiting for Hydra" is the way to go, even when you can't watch > the actual build process and thus feel that nothing is happening. > > All we need is to make sure that Hydra builds those quickly enough. > > > Greets, > Ertugrul > > -- > Ertugrul Söylemez > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On Thu, 5 Jun 2014 21:01:59 +0100 Shell Turner wrote: > So is the argument that it should be possible to update the channel > with the new package definition before the binary cache has finished > building, thus letting people rebuild their systems locally if need > be? That seems reasonable. I think a nice solution would be to add build priorities to Hydra. When a security update is required quickly, then update the OpenSSL expression, assign a high build priority to OpenSSL and the common server packages and let Hydra do the building. Most people will build on weaker machines, so I think that "waiting for Hydra" is the way to go, even when you can't watch the actual build process and thus feel that nothing is happening. All we need is to make sure that Hydra builds those quickly enough. Greets, Ertugrul -- Ertugrul Söylemez ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
No the argument is currently this pull request, where you can force the system to use a particular package (under some condition) without doing a full rebuild: https://github.com/NixOS/nixpkgs/pull/2837 On Thu, Jun 5, 2014 at 10:01 PM, Shell Turner wrote: > So is the argument that it should be possible to update the channel > with the new package definition before the binary cache has finished > building, thus letting people rebuild their systems locally if need > be? That seems reasonable. > > For the moment, though, checking out the release-14.04 branch from git > and building from that is exactly equivalent. > > Shell > > On 5 June 2014 20:05, Luca Bruno wrote: > > No, it's not too early. Other distros immediately packaged the new > version > > and provided it in their security channel. > > It's never too early when it concerns security. > > > > > > On Thu, Jun 5, 2014 at 8:04 PM, Peter Simons wrote: > >> > >> Hi Luca, > >> > >> > It takes too much time to deliver the new packages from the nixos > >> > channel, and it would take equally long to compile them on production > >> > servers. > >> > >> that OpenSSL update was committed 5 hours ago. Isn't it a wee bit early > >> to say that the update takes "too much time"? > >> > >> Also, note that you don't have to wait for the channel to update to get > >> binaries. Running > >> > >> $ nix-build nixos -A system -I nixpkgs=$PWD --dry-run --option > >> binary-caches http://hydra.nixos.org > >> > >> in a checked-out copy of the release-14.04 branch shows that a good > >> portion of Nixpkgs has been compiled by Hydra already, and compiling the > >> rest locally is not a serious problem, IMHO. > >> > >> I agree that the ability to make quick-and-dirty replacements of core > >> libraries in a running system would be nice to have. Personally, I doubt > >> I'd ever bother with that kind of hackery though, because the normal > >> update channels are quick enough, IMHO. > >> > >> Best regards, > >> Peter > >> > >> ___ > >> nix-dev mailing list > >> nix-dev@lists.science.uu.nl > >> http://lists.science.uu.nl/mailman/listinfo/nix-dev > > > > > > > > > > -- > > www.debian.org - The Universal Operating System > > > > ___ > > nix-dev mailing list > > nix-dev@lists.science.uu.nl > > http://lists.science.uu.nl/mailman/listinfo/nix-dev > > > -- www.debian.org - The Universal Operating System ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
So is the argument that it should be possible to update the channel with the new package definition before the binary cache has finished building, thus letting people rebuild their systems locally if need be? That seems reasonable. For the moment, though, checking out the release-14.04 branch from git and building from that is exactly equivalent. Shell On 5 June 2014 20:05, Luca Bruno wrote: > No, it's not too early. Other distros immediately packaged the new version > and provided it in their security channel. > It's never too early when it concerns security. > > > On Thu, Jun 5, 2014 at 8:04 PM, Peter Simons wrote: >> >> Hi Luca, >> >> > It takes too much time to deliver the new packages from the nixos >> > channel, and it would take equally long to compile them on production >> > servers. >> >> that OpenSSL update was committed 5 hours ago. Isn't it a wee bit early >> to say that the update takes "too much time"? >> >> Also, note that you don't have to wait for the channel to update to get >> binaries. Running >> >> $ nix-build nixos -A system -I nixpkgs=$PWD --dry-run --option >> binary-caches http://hydra.nixos.org >> >> in a checked-out copy of the release-14.04 branch shows that a good >> portion of Nixpkgs has been compiled by Hydra already, and compiling the >> rest locally is not a serious problem, IMHO. >> >> I agree that the ability to make quick-and-dirty replacements of core >> libraries in a running system would be nice to have. Personally, I doubt >> I'd ever bother with that kind of hackery though, because the normal >> update channels are quick enough, IMHO. >> >> Best regards, >> Peter >> >> ___ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev > > > > > -- > www.debian.org - The Universal Operating System > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
No, it's not too early. Other distros immediately packaged the new version and provided it in their security channel. It's never too early when it concerns security. On Thu, Jun 5, 2014 at 8:04 PM, Peter Simons wrote: > Hi Luca, > > > It takes too much time to deliver the new packages from the nixos > > channel, and it would take equally long to compile them on production > > servers. > > that OpenSSL update was committed 5 hours ago. Isn't it a wee bit early > to say that the update takes "too much time"? > > Also, note that you don't have to wait for the channel to update to get > binaries. Running > > $ nix-build nixos -A system -I nixpkgs=$PWD --dry-run --option > binary-caches http://hydra.nixos.org > > in a checked-out copy of the release-14.04 branch shows that a good > portion of Nixpkgs has been compiled by Hydra already, and compiling the > rest locally is not a serious problem, IMHO. > > I agree that the ability to make quick-and-dirty replacements of core > libraries in a running system would be nice to have. Personally, I doubt > I'd ever bother with that kind of hackery though, because the normal > update channels are quick enough, IMHO. > > Best regards, > Peter > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev > -- www.debian.org - The Universal Operating System ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Perhaps there is a case to be made that the hydra.nixos.org -> CloudFront delay is too long. 2014-06-05 20:04 GMT+02:00 Peter Simons : > Hi Luca, > > > It takes too much time to deliver the new packages from the nixos > > channel, and it would take equally long to compile them on production > > servers. > > that OpenSSL update was committed 5 hours ago. Isn't it a wee bit early > to say that the update takes "too much time"? > > Also, note that you don't have to wait for the channel to update to get > binaries. Running > > $ nix-build nixos -A system -I nixpkgs=$PWD --dry-run --option binary-caches > http://hydra.nixos.org > > in a checked-out copy of the release-14.04 branch shows that a good > portion of Nixpkgs has been compiled by Hydra already, and compiling the > rest locally is not a serious problem, IMHO. > > I agree that the ability to make quick-and-dirty replacements of core > libraries in a running system would be nice to have. Personally, I doubt > I'd ever bother with that kind of hackery though, because the normal > update channels are quick enough, IMHO. > > Best regards, > Peter > > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Hi Luca, > It takes too much time to deliver the new packages from the nixos > channel, and it would take equally long to compile them on production > servers. that OpenSSL update was committed 5 hours ago. Isn't it a wee bit early to say that the update takes "too much time"? Also, note that you don't have to wait for the channel to update to get binaries. Running $ nix-build nixos -A system -I nixpkgs=$PWD --dry-run --option binary-caches http://hydra.nixos.org in a checked-out copy of the release-14.04 branch shows that a good portion of Nixpkgs has been compiled by Hydra already, and compiling the rest locally is not a serious problem, IMHO. I agree that the ability to make quick-and-dirty replacements of core libraries in a running system would be nice to have. Personally, I doubt I'd ever bother with that kind of hackery though, because the normal update channels are quick enough, IMHO. Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 05/06/2014 18:41, Shea Levy wrote: > How about something like http://sprunge.us/eJOD (untested)? I don't have > time to do testing right now. > Ahah, much what I did here: https://github.com/NixOS/nixpkgs/pull/2837 ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Luca Bruno writes: > On 05/06/2014 17:49, Shea Levy wrote: >> Pass in the system derivation and use nix-env --set to switch your >> system to the resultant derivation. >> >> I have used it in the past but only for short periods while waiting for >> a rebuild. > Can't it be done in configuration.nix rather than command line? Would it > be possible somehow? I think you can use nixpkgs.config.packageOverrides for that. > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
How about something like http://sprunge.us/eJOD (untested)? I don't have time to do testing right now. On Thu, Jun 05, 2014 at 06:21:30PM +0200, Luca Bruno wrote: > On 05/06/2014 17:49, Shea Levy wrote: > > Pass in the system derivation and use nix-env --set to switch your > > system to the resultant derivation. > > > > I have used it in the past but only for short periods while waiting for > > a rebuild. > I'd like to propose a system.securityUpdates = [ pkg1 pkg2 ... ] which > will be taken in account by system.build.toplevel automatically, > so that security updates can be easily specified in configuration.nix, > instead of messing with the command line. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 06/05/2014 06:21 PM, Luca Bruno wrote: > I'd like to propose a system.securityUpdates = [ pkg1 pkg2 ... ] which > will be taken in account by system.build.toplevel automatically, > so that security updates can be easily specified in configuration.nix, > instead of messing with the command line. Sounds nice. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 05/06/2014 17:49, Shea Levy wrote: > Pass in the system derivation and use nix-env --set to switch your > system to the resultant derivation. > > I have used it in the past but only for short periods while waiting for > a rebuild. I'd like to propose a system.securityUpdates = [ pkg1 pkg2 ... ] which will be taken in account by system.build.toplevel automatically, so that security updates can be easily specified in configuration.nix, instead of messing with the command line. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 05/06/2014 17:49, Shea Levy wrote: > Pass in the system derivation and use nix-env --set to switch your > system to the resultant derivation. > > I have used it in the past but only for short periods while waiting for > a rebuild. Can't it be done in configuration.nix rather than command line? Would it be possible somehow? ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
Pass in the system derivation and use nix-env --set to switch your system to the resultant derivation. I have used it in the past but only for short periods while waiting for a rebuild. On Thu, Jun 05, 2014 at 05:44:01PM +0200, Luca Bruno wrote: > On 05/06/2014 17:16, Shea Levy wrote: > > See the replaceDependency function in nixpkgs: > > > > https://github.com/NixOS/nixpkgs/blob/80a60810ca7e59360e8babf47c4d967f108c1e46/pkgs/top-level/all-packages.nix#L407-L409 > Also, replaceDependency seems to work with a single drv, but then how to > apply the new openssl to all the drvs in pkgs? > Other than just linking it in the mailing list, are you using it in > reality Shea? > ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 05/06/2014 17:16, Shea Levy wrote: > See the replaceDependency function in nixpkgs: > > https://github.com/NixOS/nixpkgs/blob/80a60810ca7e59360e8babf47c4d967f108c1e46/pkgs/top-level/all-packages.nix#L407-L409 Also, replaceDependency seems to work with a single drv, but then how to apply the new openssl to all the drvs in pkgs? Other than just linking it in the mailing list, are you using it in reality Shea? ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
On 05/06/2014 17:16, Shea Levy wrote: > See the replaceDependency function in nixpkgs: > > https://github.com/NixOS/nixpkgs/blob/80a60810ca7e59360e8babf47c4d967f108c1e46/pkgs/top-level/all-packages.nix#L407-L409 This deserves a note in the wiki, with a practical example on how to add it, and how to remove it once it gets in the channel. I will try it and try to create a page about it. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Openssl and fast security updates
See the replaceDependency function in nixpkgs: https://github.com/NixOS/nixpkgs/blob/80a60810ca7e59360e8babf47c4d967f108c1e46/pkgs/top-level/all-packages.nix#L407-L409 On Thu, Jun 05, 2014 at 05:09:52PM +0200, Luca Bruno wrote: > This is the second time since I'm using nixos there's a need for a > really important security update. > It takes too much time to deliver the new packages from the nixos > channel, and it would take equally long to compile them on production > servers. > Are there any plans to overcome this drawback? Is there any quick fix > documented anywhere for updating a package without recompiling the world? > I'd like to open an issue about this, security is important and such > updates must be delivered soon, not after several days. > > Best regards, > ___ > nix-dev mailing list > nix-dev@lists.science.uu.nl > http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Openssl and fast security updates
This is the second time since I'm using nixos there's a need for a really important security update. It takes too much time to deliver the new packages from the nixos channel, and it would take equally long to compile them on production servers. Are there any plans to overcome this drawback? Is there any quick fix documented anywhere for updating a package without recompiling the world? I'd like to open an issue about this, security is important and such updates must be delivered soon, not after several days. Best regards, ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
