One of the banes of a Qubes addict's existence is memory. Too many times I see that red stop sign and breathe a sigh of frustration, that I need to shut down or mem-set other VM's to start up another AppVM. I like my VM separation, dammit, which means lots of VMs.
In a perfect world, I'd have a kick-ass server with 64G. In the real world, I have to make do with 4G. And in the domain of laptops (which I personally just can't trust any more, lol), memory constraints of 4G-8G are going to be present for awhile. A few questions and observations: - Have the Qubes devs ever had any thoughts of making the service VM's of sys-net and sys-firewall "headless," or generally more minimalistic? For a netcard-hosting packet slinger and a packet filter firewall VM to both have most of their memory used up by 30M X servers, pulseaudio, and so on, is a bit ironic. And exposes a lot more attack surface, yadda, yadda. (More templates is a pain, for sure, just as Whonix-Cubes uses two more templates of its own. But in the case of these important service VM's, I think the benefits of memory saving and reduced attack surface would be worth it.) Those 300M VM's add up pretty quickly on a 4G system. I think the evolution of Qubes is probably going towards having *more* service (and user) VM's; e.g. a actual storage domain, a default USB domain, maybe even a sound/keyboard/mouse domain, maybe bluetooth isolation, or whatever. (The concept of Qubes is awesome vs. a traditional metal kernel and multiple Qemu's on top, for example. But dom0 has so much in it right now, that benefit is partly theoretical at the current time, IMHO.) The more separation, and the more stuff moved out of dom0 the better, which is going to mean more VM's. If these were headless 80M (or whatever) VM's instead of honkin' 300M VM's, it becomes a *lot* more practical for those without super-machines. You'd need some way to access the VM, but the aforementioned X clients talking X protocol to an outside X server, or, even leaner, sshd would meet those needs. Any need for an xterm or X apps from these headless VM's could easily be done by the X clients connecting to another VM's (or dom0's) X server directly, without needing their own. Just a bit of iptables and X config'ing. You'd be putting more trust in ssh, but doing certificate based auth between VM's, could reduce the pain, and the ssh attack surface is probably a lot smaller than a having whole damn X server in your packet slinger/filter. I started work last night on making a headless sys-net accessible via ssh/X protocol, but got a big bogged down in iptables. I'm a bit rusty, and I need to do a good re-education of myself on iptables and hit it again (and not at 3am :P). Also, I spun my wheels for a bit because I didn't realize at first that everybody on 10.137.2.* isn't really on the same subnet in the traditional sense, but has to send everything through 10.137.2.1 sys-firewall, correct? Seems a bit odd, but I can understand the restriction from a security standpoint, and with ipt forwarding, the desired effect can be achieved--with some concentration, which I didn't have last night, lol. Actually, part of me wonders if being forced to send everything through sys-firewall for 10.137.2.*'s to talk to each other is *more* of a security risk. (If sys-firewall is compromised, it gets to see all the VM's talking to each other, which wouldn't be the case of 10.137.2.*/24 acted like a normal LAN. Plus, it'd probably be more efficient without the additional iptables relaying. But that's a secondary concern.) (- A bit of an oblique thought, might be the sharing of memory between the common pages of X server (and other app) instances in different VM's. But that strikes me as having a whole nightmare of complications to implement, along with a host of security issues. Just trimming unnecessary stuff out of the VM's seems a lot more straight-forward.) - It might make sense to have the sys-net and sys-firewall service VM's based upon a different, more minimal template than the user-app-vm-intended fedora-23. Having pulseaudio, alsa, modemmanager (what even is that??) and a bunch of other stuff running in a VM that is dedicated to just slinging packets or providing an iptables firewall seems terribly wasteful. (And I'm talking using up real resident RSS memory, not just Virtual space.) It's definitely a side-project of mine to take sys-net and sys-firewall down to the most minimal state/size possible. Saves memory, and reduces attack surface, allows for more work VM's. Plus, for the truly paranoid (me, in case you haven't picked up on that yet; although I prefer the term "cautious"), there has been talk about sound cards being as covert channels (potentially between VMs in this case). So it seems like an unnecessary security risk. Yes, in order for sys-net or sys-firewall to talk to other VM's via the sound card, you need both ends to be compromised, in which case you're kinda screwed anyway, but still, why have any attack surface exposed, and memory wasted, when you don't have to. (Actually, that's not quite true, if there's any validity to Ruiu's air-gap-jumping audio-based malware claims, then a compromised sys-vm/sys-firewall could use their audio abilities to send stuff through the Qubes systems dom0 speakers, to a listening smartphone or other air-gapped compromised device. Some of that stuff seems a bit of a stretch, but why run the risk, if you don't have a compelling need to be playing MP3's or watching YouTubes from sys-net :) Even though I'm slightly skeptical of some of his claims, I still only use headphones, not speakers, and mute dom0's volume control when not in use, for the heck of it.) - With lean, mean service-based template, I'd probably be more willing to run i2p, Freenet, a Tor relay, etc., in streamlined headless VM's. But as it stands, I just can't spare the VM space, which is kind of sad. :( - Another related question: if I have enough swap space in dom0, why a I restricted to launching VM's due to memory restrictions? I realize you'd get into swapping-hell pretty quickly if you weren't careful, and one can crank down individual VM's max memory to make them do their own internal swapping, but I'm still curious as to why VM's can't be a bit more "swappy" from dom0's standpoint. Does their physical memory (from their standpoint) have to be physical memory in dom0, period? - And slightly off-topic, but somewhat related: I like Qubes-whonix (despite my hesitation to add another trust actor on top of Redhat, Invisible Labs [you guys are cool, tho' :)]). It seems weird to me that Qubes-Whonix's whonix-gw VM talks to sys-firewall and then sys-net. whonix-gw was kind of designed to *be* the network facing VM, with it's own iptables firewall (more comprehensive than Qubes'), protecting whonix-workstation from the net. (I realize I could get rid of sys-net/sys-firewall, and add the net card to sys-whonix(gw), to have that kind of configuration, but I'm a big curious as to why Qubes-Whonix layered it this way. Requiring *four* 300M VM's just to get the web browser fired up doesn't leave much room for other VM's. I assume it was just to fit in with minimal disruption to the standard Qubes configuration, but I was curious if there was other motivations. I might be asking the wrong people on this list, but I thought I'd throw it out there.) - Finally, why does dom0 take up so much memory by default? Does it just hold onto it to dole out to the other VM's? Most of my VM's are 300M'ish (unless that pig Firefox is running with a few tabs open, in which case they shoot up to 900M or more), but dom0 is almost always 1.5G or so. mem-set'ing it down to 800M doesn't seem to harm things, and lets me start more VM's. So what's up with that? :) ---- Apologies for the length of this note. If you guys weren't so responsive and helpful, I wouldn't bother you wish so many questions! (And I'm only trying to help improve things for everyone.) Thanks! JJ -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/8d1842645405ef1ed30867823eb98070.webmail%40localhost. For more options, visit https://groups.google.com/d/optout.
