[qubes-users] Update all domains for Qubes OS 4.0

[Jimmy Axenhus]

Hi!

A while ago I wrote a small Python-script to update all VMs at once.
I've now upgraded it for Qubes OS 4.0.
At the same time I decided to rewrite it in shell to avoid relying on
the Python API. Unless something drastically changes I expect this to
also work for future Qubes OS releases.

URL:

https://gist.github.com/JimmyAx/818bcf11a14e85531516ef999c8c5765

Embedded:

#!/bin/sh

for domain in $(qvm-ls --fields NAME,CLASS | \
    awk '($2 == "TemplateVM" || $2 == "StandaloneVM") {print $1}'); do
    qvm-run --service $domain qubes.InstallUpdatesGUI
done

sudo qubes-dom0-update

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/6f604b13-a3cf-898b-e904-fb6352aeb1de%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] Programmatically performing software update

[Jimmy Axenhus]

Den 2017-03-01 kl. 06:41, skrev loke...@gmail.com:
> I tend to use Qubes VM's not just for security purposes, but also as a very 
> convenient way to have clean, manageable VM's where I only install the stuff 
> I need for a specific task.
> 
> This has led me to have a large number non-template VM's, all of which needs 
> to be manually updated every time there is a software update.
> 
> Right now, for each VM, I have to click on update, wait for the dialogue box 
> to spin for a while waiting for the VM to start, then manually choose “yes” 
> to update. This process gets old really fast.
> 
> Is there a way to automate this so that I can simply say “yes, I want to 
> update all these VM's, go ahead and do it”?
> 
> Regards,
> Elias
> 

I got a Python script for that. Put in dom0 and run it. Unlike Andrews
script this one will update all VMs in parallel and is interactive
rather than non-interactive.

https://gist.github.com/JimmyAx/818bcf11a14e85531516ef999c8c5765


Here it is, embedded in the email:



#!/usr/bin/python2

import subprocess

from time import sleep

from qubes.qubes import QubesVmCollection


if __name__ == "__main__":
qvm_collection = QubesVmCollection()
qvm_collection.lock_db_for_reading()
try:
qvm_collection.load()
finally:
qvm_collection.unlock_db()

vms = qvm_collection.values()

exclude_vms = ("debian-8-hvm", "win7", "win7-orig")
#"archlinux-aur", "archlinux")

processes = []
for vm in vms:
dom0 = vm.qid == 0
if not dom0 and vm.updateable and vm.name not in exclude_vms:
print "Updating VM template %s..." % vm.name
if not vm.is_running():
vm.start()
sleep(3)
p = vm.run_service("qubes.InstallUpdatesGUI", gui=True,
wait=False, passio_popen=True)
processes.append((vm, p))

print "Waiting for VMs to complete updating..."
for vm, p in processes:
p.wait()
if vm.is_running():
vm.shutdown()

subprocess.check_call(["sudo", "/usr/bin/qubes-dom0-update"])

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/9150dbaa-6783-f57c-611a-5bef296da163%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] Re: Redshift in dom0?

[Jimmy Axenhus]

Den 2017-01-24 kl. 18:34, skrev ms.amne...@donotusemy.info:
> On Monday, January 23, 2017 at 4:22:56 PM UTC+1, Kopimi Security wrote:
> 
> The only "tweak" I still need is the mouse pointer, as it is unaffected by 
> redshift on my Qubes.
> 
> Anyone any ideas about that?
> 


You need to disable the hardware cursor. It's a known limitation. [1]

I got this in /etc/X11/xorg.conf and it's doing the job nicely.

Section "Device"
Identifier "Device0"
Driver "radeon"   # Replace with your driver
Option "SWCursor" "on"
EndSection


[1]: http://jonls.dk/redshift/

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/b5990689-cb20-04b2-98e5-cb4732e75a01%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] dom0 window manager style

[Jimmy Axenhus]

Den 2016-11-13 kl. 20:49, skrev cubit:
> I have a really stupid question.  I was looking at dom0's window manager
> settings and changed the style and now I can not find the one that Qubes
> uses by default.   Can anyone know what it is called or how to get it
> back, I've gone though the list and nothing looks similar and yes I have
> tried "Default" :)
> 

If you're using XFCE I think it's Nodoka. (That's what I got anyway.)

> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to qubes-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to qubes-users@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-users/KWUE8cq--3-0%40tutanota.com 
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/e6321a3e-ff38-d902-df4f-d4a131485859%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] Issue #2407: Renaming VM works partially

[Jimmy Axenhus]

Den 2016-11-08 kl. 22:43, skrev Marek Marczykowski-Górecki:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> On Tue, Nov 08, 2016 at 08:00:30PM +0100, Jimmy Axenhus wrote:
>> https://github.com/QubesOS/qubes-issues/issues/2407
>>
>> Sadly I reproduced this by accident. I renamed a VM, but it somehow
>> failed. It did not get renamed in Qubes VM Manager, yet it did in
>> /var/lib/qubes/appvms and the start menu in dom0. A simple reboot did
>> not solve the problem as Qubes VM Manager still shows the old name only.
>> I can not rename the old VM again ("file doesn't exist"-error).
>>
>> Is there any way to rename the entry in Qubes VM Manager or do I need to
>> rename the new VM in /var/lib/qubes/appvms to its old name and try again?
> You can try by editing /var/lib/qubes/qubes.xml - change "name",
> "dir_path" and "conf_file" properties - or simply all occurrences of old
> name. Make copy of this file first, just in case.
That worked without any problems and the VM is starting up fine. Thanks!
>
>> I have not been able to reproduce it again by creating a few new VMs and
>> renaming them.
> Check .xsession-errors or .xsession-errors.old for some Qubes Manager
> error - it would really help with debugging this.
Nothing interesting in those files. I did move them to a different
folder using Ctrl + Alt + F2 before I logged in to the desktop so I
think they were not rotated. I'll check those files next time if I run
into the bug again.
>
> - -- 
> Best Regards,
> Marek Marczykowski-Górecki
> Invisible Things Lab
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2
>
> iQEcBAEBCAAGBQJYIkcJAAoJENuP0xzK19csdFkH/iH+Og1VejUpFzQVlM9fCc35
> wDYVSpCLky8p0e4KOdtVDqfhxnUg6KHH0k3Ej0Bve0y+gUO3z67vEEfXjXgW2TuF
> qPKm+DcY+ABb/6lj5pDodeSmrQbBnxTeaT9PlLLutelmVlvGswI0V3WXaEz7JvAl
> tOXTZcgSETeeMPb7TXhUAbzBonYquTjTk9ruWpwaitflLOLpOsQ7mC4CtC+9wSTx
> 60lhQdavPmq2J3YV2xPx+YJC72ARr4AUX8IsuA1AM3aPSXD+lcSsZcdMaFZEV1tu
> YfCn4JUfG6rkgipAPB+VV5I96gncymGjWp21uVzXyw60ZTpyoipUpCDfhXqs9Es=
> =HUAt
> -END PGP SIGNATURE-
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/d65abbf1-e569-3d5e-2643-f5578937ea80%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

[qubes-users] Issue #2407: Renaming VM works partially

[Jimmy Axenhus]

https://github.com/QubesOS/qubes-issues/issues/2407

Sadly I reproduced this by accident. I renamed a VM, but it somehow
failed. It did not get renamed in Qubes VM Manager, yet it did in
/var/lib/qubes/appvms and the start menu in dom0. A simple reboot did
not solve the problem as Qubes VM Manager still shows the old name only.
I can not rename the old VM again ("file doesn't exist"-error).

Is there any way to rename the entry in Qubes VM Manager or do I need to
rename the new VM in /var/lib/qubes/appvms to its old name and try again?

I have not been able to reproduce it again by creating a few new VMs and
renaming them.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/66d4df4e-a588-6f3d-673f-5e226b8da7fd%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] Archlinux template build fails: 'readdir_r' is deprecated

[Jimmy Axenhus]

Den 2016-10-09 kl. 11:05, skrev Dimitri:
> I didn't build the archlinux template for a long time. Once it worked as 
> described here https://www.qubes-os.org/doc/templates/archlinux/ but now it 
> fails with this error (for 3.1 and 3.2): 
> 
> libxl_utils.c: In function 'libxl__remove_directory':
> libxl_utils.c:480:9: error: 'readdir_r' is deprecated 
> [-Werror=deprecated-declarations]
>  int r = readdir_r(d, de_buf, &de);
>  ^~~
> In file included from libxl_internal.h:25:0,
>  from libxl_utils.c:20:
> /usr/include/dirent.h:183:12: note: declared here
>  extern int readdir_r (DIR *__restrict __dirp,
> ^
> cc1: all warnings being treated as errors
> make[3]: *** 
> [/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools/libxl/../../tools/Rules.mk:107:
>  libxl_utils.o] Error 1
> make[3]: Leaving directory 
> '/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools/libxl'
> make[2]: *** 
> [/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools/../tools/Rules.mk:123: 
> subdir-install-libxl] Error 2
> make[2]: Leaving directory '/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools'
> make[1]: *** 
> [/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools/../tools/Rules.mk:118: 
> subdirs-install] Error 2
> make[1]: Leaving directory '/home/user/qubes-src/vmm-xen/src/xen-4.6.1/tools'
> make: *** [Makefile:102: install-tools] Error 2
> ==> ERROR: A failure occurred in build().
> Aborting...
> /home/user/qubes-builder/qubes-src/builder-archlinux/Makefile.archlinux:120: 
> recipe for target 'dist-package' failed
> make[2]: *** [dist-package] Error 2
> 
> 
> How can I fix this? Ignoring the deprecation? Replace readdir_r with a non 
> deprecated alternative?
> 
> Thanks for help!
> 

I built the archlinux template yesterday and ran into the same problem.
I had to modify /etc/makepkg.conf and add "-Wno-deprecated-declarations"
to CFLAGS.

That's not enough however as I also had to bind mount /dev/pts on the
chroot because gpg was failing when installing packages. I poked around
in various scripts and somehow got to it work.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/4664a78d-cbee-b834-8db4-2ed667f73f9b%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] disposableVM not really disposable

[Jimmy Axenhus]

Den 2016-08-05 kl. 15:49, skrev HW42:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Marek Marczykowski-Górecki:

On Fri, Aug 05, 2016 at 11:57:38AM +0200, Marek Marczykowski-Górecki wrote:

On Wed, Jul 27, 2016 at 10:38:52AM +0100, cubit wrote:

26. Jul 2016 23:14 by marma...@invisiblethingslab.com:


Check file list in /var/lib/qubes/appvms/fedora-23-dvm. Especially date
and permissions of volatile.img there.






It's actually fedora-23-stable-dvm based off a copy of the template




Here is the file list,  there is one noticeable difference when
comparing it to fedora-23-dvm folder.For my template
volatile.img is chmod 600  while it is 644 in fedora-23-dvm.
Changing the permissions to match 644 does not change anything, the
problem still exists


@cubit: What is the output of 'sudo sh -c umask' in dom0?


Change it to 664 (same as other files there) - so qubes group will have
write access.


What are permissions on the directory itself?



(...)



-rw-rw-r-- 1 user qubes1669 Jun  2 20:15 disp8.conf
-rw-rw-r-- 1 user qubes1670 Jun  2 20:15 disp9.conf
-rw-rw-r-- 1 user qubes   309223442 Jul 21 09:49 dvm-savefile
-rw-rw-r-- 1 user qubes1925 Jul 21 09:49 fedora-23-stable-dvm.conf
lrwxrwxrwx 1 user qubes  55 Mar 31 16:15 icon.png -> 
/usr/share/icons/hicolor/128x128/devices/appvm-gray.png
-rw-rw-r-- 1 user qubes   1 Jul 21 09:49 netvm-id.txt
-rw-rw 1 user qubes  2147483648 Jul 21 09:49 private.img
-rw-rw-r-- 1 user qubes1024 Jul 21 09:49 saved-cows.tar
-rw--- 1 root qubes 12348030976 Jul 26 20:00 volatile.img
-rw-rw-r-- 1 user qubes  70 Mar 31 16:15 whitelisted-appmenus.list


One case in which this happens is when the root user has a restrictive
umask. Then prepare-volatile-img.sh creates a file as show above.
qubes-prepare-saved-domain.sh then fails siletly to include volatile.img
into saved-cows.tar.

This can be simply fixed by not running prepare-volatile-img.sh as root
(see attached patch, sudo is no longer necessary there).

But I'm not sure if this is the problem in the reported cases.
Especially this should be rather easily noticed since you can't start
a dispvm when another is still running.

What are the reasons for using bsdtar instead of tar? I tried to make
the script to fail in this case but bsdtar justs prints a warning on
permission errors and still exits with 0 (in contrast to GNU tar).



I just ran into this bug too. First time ever. Fired up a disposable VM 
and discovered that there was some history left from another one I had 
running earlier. I tried starting another disposable VM again without 
closing the buggy one, but it won't start. I know that behavior is 
another bug, but perhaps they are related somehow? I should note that 
the current uptime is 6-7 hours, yet I've had longer uptimes than that 
and not run into this bug before.


I know the workarounds are to regenerate the disposable VM file and to 
reboot the computer to solve the VMs not being created.


I've collected some information, but I don't know if it will help. If 
it's relevant the last disposable VM that started successfully was the 
one with the leftover history and it has the name disp6. Anything else 
that might be interesting before I reboot?





[user@dom0 ~]$ file /var/lib/qubes/appvms/fedora-23-dvm/
/var/lib/qubes/appvms/fedora-23-dvm/: setgid, directory
[user@dom0 ~]$ ls /var/lib/qubes/appvms/fedora-23-dvm/
disp10.conf  disp2.conf  disp4.conf  disp6.conf  disp8.conf 
dvm-savefileicon.png  private.img whitelisted-appmenus.list
disp1.conf   disp3.conf  disp5.conf  disp7.conf  disp9.conf 
fedora-23-dvm.conf  netvm-id.txt  saved-cows.tar  volatile.img

[user@dom0 ~]$ ls -lh /var/lib/qubes/appvms/fedora-23-dvm/
/var/lib/qubes/appvms/fedora-23-dvm/:
totalt 436M
-rw-rw-r-- 1 user qubes 1,7K  5 aug 19.30 disp10.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 13.56 disp1.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 14.14 disp2.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 15.21 disp3.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 15.32 disp4.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 15.40 disp5.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 19.51 disp6.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 19.56 disp7.conf
-rw-rw-r-- 1 user qubes 1,7K  6 aug 19.57 disp8.conf
-rw-rw-r-- 1 user qubes 1,7K  5 aug 18.54 disp9.conf
-rw-rw 1 user qubes 295M  6 aug 12.12 dvm-savefile
-rw-rw-r-- 1 user qubes 1,9K  6 aug 12.12 fedora-23-dvm.conf
lrwxrwxrwx 1 user qubes   55 28 jul 16.42 icon.png -> 
/usr/share/icons/hicolor/128x128/devices/appvm-gray.png

-rw-rw 1 user qubes1  6 aug 12.12 netvm-id.txt
-rw-rw 1 user qubes 2,0G  6 aug 12.12 private.img
-rw-rw 1 user qubes 1,0K  6 aug 12.12 saved-cows.tar
-rw-rw-r-- 1 user qubes   70 28 jul 16.42 whitelisted-appmenus.list
-rw--- 1 root qubes  12G  6 aug 20.02 volatile.img
[user@dom0 ~]$ date 



lör aug  6 20:02:50 CEST 2016 




 I paused here for a while before checking again 

[user@dom0 ~]$ l

Re: [qubes-users] How to properly administer several templates?

[Jimmy Axenhus]

Den 2016-06-16 kl. 19:23, skrev Andrew David Wong:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 2016-06-15 09:33, Jimmy Axenhus wrote:

Den 2016-06-15 kl. 11:14, skrev Andrew David Wong:

On 2016-06-15 01:56, Franz wrote:

On Tue, Jun 14, 2016 at 10:49 PM, Andrew David Wong
 wrote:

On 2016-06-14 15:20, Albin Otterhäll wrote:

I've eight different templates (copies, etc.) and
administrate them takes quite a while. How should I
streamline this and make it as easily as possible to
update and install applications across several
templates?


Personally, I found that the vast majority of time I spent
administering templates was spent on keeping them updated, so I
wrote a bash script to update them and set it to run
periodically as a cron job in dom0.



And what happens when the update process stops for some
reason, such as for being unable to verify packages or for
suggesting some manual steps?

Then it'll just fail to update. I log all the output and check it
periodically, so if something requires manual interaction, I just
do it manually at some later time. Most of the time, no manual
interaction is required for standard updating.


Would you mind sharing that bash script? It sounds way more
convenient than manually updating all the templates from the Qubes
VM Manager.


I'm sure it's suboptimal in many ways, so if you (or anyone else) sees
room for improvement, feel free to let me know. :)

As a GitHub Gist (with some syntax highlighting):

https://gist.github.com/andrewdavidwong/d0b109186de65835255d467ae103c289

As plain text:

##

#!/bin/bash
# Set the updatevm.
updatevm=sys-firewall
# Declare arrays of VMs to be updated.
Fedora=(
'fedora-23'
'fedora-23-minimal'
)
Debian=(
'whonix-gw'
'whonix-ws'
)
# Proceed only if the UpdateVM is running.
if qvm-ls $updatevm | grep -q Running; then
 echo "Starting update process."
 # Download dom0 updates.
 echo "Downloading updates for dom0 at $(date -Is) ...";
 sudo qubes-dom0-update -y;
 sleep 5;
 # Download Fedora VM updates.
 for vm in ${Fedora[*]}; do
 if qvm-ls $vm | grep -q Halted; then
 echo "Updating $vm at $(date -Is) ...";
 qvm-start --no-guid -q $vm;
 sleep 3;
 qvm-run -a --nogui -p -u root $vm \
 'dnf -y --refresh upgrade';
 sleep 10;
 qvm-shutdown -q --wait $vm;
 sleep 3;
 fi
 done
 # Download Debian VM updates.
 for vm in ${Debian[*]}; do
 if qvm-ls $vm | grep -q Halted; then
 echo "Updating $vm at $(date -Is) ...";
 qvm-start --no-guid -q $vm;
 sleep 3;
 qvm-run -a --nogui -p -u root $vm \
 'apt-get -y update && apt-get -y dist-upgrade';
 sleep 10;
 qvm-shutdown -q --wait $vm;
 sleep 3;
 fi
 done
 echo "Finished update process at $(date -Is).";
fi
exit

##

- -- 
Andrew David Wong (Axon)

Community Manager, Qubes OS
https://www.qubes-os.org
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXYuCYAAoJENtN07w5UDAwCR8P/iTbzgmV565vwtS6b9E+pAq7
TMvK2mRECwvhbuqSOCmCQqIkxXHkgjMfWUWRZowxA+BmmeS6fhUbQFiF7k/XoS/Y
D5uxP9txtmAlyFRQGtGvlylG9+0QzSfaGHyxxabJcbEMlh/6NEs/BplpJX9CwqqQ
uf6YIXlNCkuaP4wDEMdLg0eUrKb9GwicdWTxL4SGjZ9wK1t3RzbTtMHghBAsdr6Y
opx655TpmPW/1oG+K+CgGKEu6e1QwBDhmusRl9AxKLOdDUq9cqxeLycK7CZ6fBXQ
G4Ptbl5wTfsaDKwDsmM/QAoaC0BtkTjgwZmCvVT9YFcDz4+WClMRuL6mS8Ftx/cl
nF5xXaKg2F+sM08lZSAU2SvNlmJFygMGlbNFQpPatxw1KH/w4tXGguQFrrUQFzMm
pn8P7ThZSuCrCQ5xj3t1LRJFQuK712sGiR2xzlY9VB32uCr4YC+27Cm1IftGhoLM
ogo7UseqeiybPkpjB3JFOvXqMHvf8n//aeCs/ZIIUheWD2T1LCCd2DeRZ/rBiesX
XRsuwKJA+zAFh89MN+BRwYRf8elNVkLolM83++MSqA28BzkSrGbDes/SmG8svRUN
gNxIOsnzs04tvFyuoMmIeRrOBMDwBm7gbuBUkBJWyTBfO/SBF/NdMRil2k2C9IJI
JKZfz/5hkCQIaQpqprpZ
=o+UX
-END PGP SIGNATURE-



Now it's my turn!
Inspired by your script and the fact that it will probably be a few 
months until Qubes 3.2 is released I decided to go ahead and try a 
cleaner version of it. After looking at how the VM Manager does it I 
ended up with a completely rewritten script in Python. The difference is 
that this one is interactive (I prefer that) and it's starting them all 
right after each other. In other words you might get a bunch of windows 
popping up.


I have no idea how stable it is or if it's going to work in Qubes 3.2, 
but it works fine in 3.1.


License is CC-0 or GPLv2+. Pick the one you like.

https://gist.github.com/JimmyAx/818bcf11a14e85531516ef999c8c5765

###

#!/usr/bin/python2

import subprocess

from qubes.qubes import QubesVmCollection


if __name__ == "__main__":
qvm_collection = QubesVmCollection()
qvm_collection.lock_db_for_reading()
try:
qvm_collection.load()
finally:
qvm_collection.unlock_db()

vms = qvm_collection.val

Re: [qubes-users] Qubes VM Manager becomes unresponsive for a while when trying to backup (excessive disk IO)

[Jimmy Axenhus]

Den 2016-06-20 kl. 22:52, skrev Marek Marczykowski-Górecki:

On Mon, Jun 20, 2016 at 10:39:30PM +0200, Jimmy Axenhus wrote:

Hi!

Whenever I try to make a backup with the Qubes VM Manager (System -> Backup
VMs) it appears to be freezing and I have to wait several minutes for it to
start responding again. I've been able to make backups before, but after not
having done it for a while I decided to go ahead and do it again. Before it
was really fast and worked without any issues. The VM Manager is not
completely stuck though as I can see the CPU going up and down using top.

Investigation of qubes-manager by inspecting the stack by modifying[1]
/usr/bin/qubes-manager and using iotop shows that the process is spending
most of it's time reading from the disk. qubes-manager seems to spending
most of it's time in /usr/lib64/python2.7/site-packages/qubes/qubesutils.py,
function get_disk_usage(). The stack trace I got indicated that it was
calling os.walk(). The path it was scanning was one which I know is on a
secondary slow HDD and contains *plenty* of subdirectories and files. Looks
like this is causing the problem.

I use an SSD for Qubes, but got a secondary HDD attached to dom0. The HDD is
being mounted automatically on boot. When it worked fine I only had the SSD.

If I wait long enough it will abort the backup attempt and become responsive
again, but that's because of some permission issues on the contents of my
HDD that I need to fix.

It is probably mounted somewhere inside your dom0 home. When you select
to backup it (it is by default), manager needs to calculate its disk
usage. If you want to skip this, simply don't select "dom0" to backup.


I moved the HDD to /media/hdd and that solved it! The GUI is no longer 
freezing or becoming unresponsive and its now as fast as it used to be. 
Guess I shouldn't be mounting disks inside /home in the future.


Thank you for your help!




It would be much better if the Qubes VM Manager GUI didn't freeze and
instead provided some kind of feedback of what it is doing.

Yes, space calculation (unlike performing backup itself) is running in
the GUI thread. That space calculation shouldn't normally take much
time...


[1]: 
http://stackoverflow.com/questions/132058/showing-the-stack-trace-from-a-running-python-application




--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/56172915-04a4-44c5-e056-297abe26440f%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

[qubes-users] Qubes VM Manager becomes unresponsive for a while when trying to backup (excessive disk IO)

[Jimmy Axenhus]

Hi!

Whenever I try to make a backup with the Qubes VM Manager (System -> 
Backup VMs) it appears to be freezing and I have to wait several minutes 
for it to start responding again. I've been able to make backups before, 
but after not having done it for a while I decided to go ahead and do it 
again. Before it was really fast and worked without any issues. The VM 
Manager is not completely stuck though as I can see the CPU going up and 
down using top.


Investigation of qubes-manager by inspecting the stack by modifying[1] 
/usr/bin/qubes-manager and using iotop shows that the process is 
spending most of it's time reading from the disk. qubes-manager seems to 
spending most of it's time in 
/usr/lib64/python2.7/site-packages/qubes/qubesutils.py, function 
get_disk_usage(). The stack trace I got indicated that it was calling 
os.walk(). The path it was scanning was one which I know is on a 
secondary slow HDD and contains *plenty* of subdirectories and files. 
Looks like this is causing the problem.


I use an SSD for Qubes, but got a secondary HDD attached to dom0. The 
HDD is being mounted automatically on boot. When it worked fine I only 
had the SSD.


If I wait long enough it will abort the backup attempt and become 
responsive again, but that's because of some permission issues on the 
contents of my HDD that I need to fix.


It would be much better if the Qubes VM Manager GUI didn't freeze and 
instead provided some kind of feedback of what it is doing.



[1]: 
http://stackoverflow.com/questions/132058/showing-the-stack-trace-from-a-running-python-application


--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/ca993bd6-6cd2-69f7-722d-e31b389193ce%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.

Re: [qubes-users] How to properly administer several templates?

[Jimmy Axenhus]

Den 2016-06-15 kl. 11:14, skrev Andrew David Wong:

On 2016-06-15 01:56, Franz wrote:

On Tue, Jun 14, 2016 at 10:49 PM, Andrew David Wong
 wrote:

On 2016-06-14 15:20, Albin Otterhäll wrote:

I've eight different templates (copies, etc.) and
administrate them takes quite a while. How should I
streamline this and make it as easily as possible to update
and install applications across several templates?



Personally, I found that the vast majority of time I spent
administering templates was spent on keeping them updated, so I
wrote a bash script to update them and set it to run periodically
as a cron job in dom0.



And what happens when the update process stops for some reason,
such as for being unable to verify packages or for suggesting
some manual steps?




Then it'll just fail to update. I log all the output and check it
periodically, so if something requires manual interaction, I just do
it manually at some later time. Most of the time, no manual
interaction is required for standard updating.

Would you mind sharing that bash script? It sounds way more convenient 
than manually updating all the templates from the Qubes VM Manager.


--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/e4b8e71c-5f5f-17d7-5e16-3652f1fccaf4%40axenhus.com.
For more options, visit https://groups.google.com/d/optout.