Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread Volth 
They are not holes if you mean security, they are stability and data
integrity issues we regulary step into as server load and number of
files grow.

The fourth issue is https://lkml.org/lkml/2016/11/24/721, it has not
its own nixpkgs ticket, discussed in
https://github.com/NixOS/nixpkgs/issues/23957 's comments.

On 3/21/17, Jookia <166...@gmail.com> wrote:
> On Tue, Mar 21, 2017 at 11:29:09PM +0100, Profpatsch wrote:
>> On 17-03-20 10:27pm, Volth wrote:
>> > Recently few bugs in 9P were found (#23957 #23020 #22695) which
>> > reveals that 9P code is not very mature and perhaps NixOS is the first
>> > team which uses 9P heavily and relies on it in production.
>>
>> Could you please provide links? 9P is a protocol,
>> do you mean holes in the protocol have been found?
>
> https://github.com/NixOS/nixpkgs/issues/23957
> https://github.com/NixOS/nixpkgs/issues/23020
> https://github.com/NixOS/nixpkgs/issues/22695
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread Jookia 
On Tue, Mar 21, 2017 at 11:29:09PM +0100, Profpatsch wrote:
> On 17-03-20 10:27pm, Volth wrote:
> > Recently few bugs in 9P were found (#23957 #23020 #22695) which
> > reveals that 9P code is not very mature and perhaps NixOS is the first
> > team which uses 9P heavily and relies on it in production.
> 
> Could you please provide links? 9P is a protocol,
> do you mean holes in the protocol have been found?

https://github.com/NixOS/nixpkgs/issues/23957
https://github.com/NixOS/nixpkgs/issues/23020
https://github.com/NixOS/nixpkgs/issues/22695
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread Profpatsch 
On 17-03-20 10:27pm, Volth wrote:
> Recently few bugs in 9P were found (#23957 #23020 #22695) which
> reveals that 9P code is not very mature and perhaps NixOS is the first
> team which uses 9P heavily and relies on it in production.

Could you please provide links? 9P is a protocol,
do you mean holes in the protocol have been found?

-- 
Proudly written in Mutt with Vim on NixOS.
Q: Why is this email five sentences or less?
A: http://five.sentenc.es
May take up to five days to read your message. If it’s urgent, call me.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread Eelco Dolstra 
Hi,

On 03/20/2017 11:27 PM, Volth wrote:

> 9P is used by NixOS to share host's nix store with Qemu virtual
> machines. Such technique is used in the build process, in the
> test-driver, so to say in the critical places.
> Recently few bugs in 9P were found (#23957 #23020 #22695) which
> reveals that 9P code is not very mature and perhaps NixOS is the first
> team which uses 9P heavily and relies on it in production.

I wouldn't call using it in tests "production" :-)

> Shouldn't we replace 9P with something battle-tested like NFS or Samba?

We used to use Samba/CIFS for the VM tests, but it was slower than 9p [1],
required TCP/IP at an early stage in the initrd, and required babysitting a
Samba process, whereas 9p is built into QEMU. It was also not super-reliable
(e.g. CIFS mounting had a tendency to time out if the host was under high load).

A while ago I played with Ganesha NFS but couldn't get it to work. It's probably
doable but much more complicated than passing a couple of flags to QEMU.

[1] 
https://github.com/NixOS/nixpkgs/commit/0e386d0c1373809d3459d09532ad36be0faf6dfb

> It may also improve the performance because 9P server works in qemu
> process, in user mode and there are as many servers as virtual
> machines running.

It would probably be slower, requiring more context switches between the VM,
QEMU and the FS server. Also, 9p uses the virtio transport which is ideal for VM
<-> host communication.

Vladimir wrote:

>> In terms of performance getting rid of QEMU where possible is probably
>> the better option.  Containers are fairly mature these days, and then
>> sharing file-systems is a matter of bind-mounting.
> 
> For some tests containers certainly aren't enough, as we do want to test
> booting, the kernel, etc.

Also, running a NixOS system requires a full UID namespace, whereas currently
Nix builds only get a single UID. For tests that don't require testing the
kernel or initrd, running them as a container would be great though.

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread Vladimír Čunát 
On 03/21/2017 12:37 AM, Ertugrul Söylemez wrote:
> In terms of performance getting rid of QEMU where possible is probably
> the better option.  Containers are fairly mature these days, and then
> sharing file-systems is a matter of bind-mounting.

For some tests containers certainly aren't enough, as we do want to test
booting, the kernel, etc.




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-21 Thread zimbatm 
What matters are that the tests are valid, reproducible   and not too slow.
If you want to replace the filesystem and it improves any of these
qualities I don't see any problems :)

On Mon, 20 Mar 2017, 23:37 Ertugrul Söylemez,  wrote:

> > 9P is used by NixOS to share host's nix store with Qemu virtual
> > machines. Such technique is used in the build process, in the
> > test-driver, so to say in the critical places.
> > Recently few bugs in 9P were found (#23957 #23020 #22695) which
> > reveals that 9P code is not very mature and perhaps NixOS is the first
> > team which uses 9P heavily and relies on it in production.
> >
> > Shouldn't we replace 9P with something battle-tested like NFS or
> > Samba?  It may also improve the performance because 9P server works in
> > qemu process, in user mode and there are as many servers as virtual
> > machines running.
>
> In terms of performance getting rid of QEMU where possible is probably
> the better option.  Containers are fairly mature these days, and then
> sharing file-systems is a matter of bind-mounting.
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Should we drop 9P?

2017-03-20 Thread Ertugrul Söylemez 
> 9P is used by NixOS to share host's nix store with Qemu virtual
> machines. Such technique is used in the build process, in the
> test-driver, so to say in the critical places.
> Recently few bugs in 9P were found (#23957 #23020 #22695) which
> reveals that 9P code is not very mature and perhaps NixOS is the first
> team which uses 9P heavily and relies on it in production.
>
> Shouldn't we replace 9P with something battle-tested like NFS or
> Samba?  It may also improve the performance because 9P server works in
> qemu process, in user mode and there are as many servers as virtual
> machines running.

In terms of performance getting rid of QEMU where possible is probably
the better option.  Containers are fairly mature these days, and then
sharing file-systems is a matter of bind-mounting.


signature.asc
Description: PGP signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev