-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Mon, Jul 10, 2017 at 09:38:22PM -0700, Andrew Morgan wrote: > Hello everyone, it's that time of the week again. > > On this episode: tests, file managing, something else?!?! > > Grab it with format here: > https://blog.amorgan.xyz/gsoc-weekly-progress-report-6.html > > Otherwise text-only is below: > > --- > > Hello again! Few different things going on this week so let's just jump > right in. > > ## Tests > > First off a basic set of tests have been written for the > `qvm-file-trust` tool. So far these are all unit tests, and mainly test > sending input to each method in `qvm-file-trust`, ensuring that certain > methods are called with the correct parameters, and anything that is > returned contains the correct data. > > In the process of writing these tests I uncovered a few bugs and quirks > here and there in the code, which is awesome because that's exactly what > tests are designed to do. Currently I only test each method directly, > however in the long run there will be tests that send varied input > externally to the program. These will check the output as well as the > trust status of any files that may be involved. > > As was evident from my last post, I had a bit of trouble getting used to > some 'pythonic' quirks, as well as the whole concept of mocking. > Nevertheless, after getting a few fundamentals down on how to feed in > fake inputs and data, writing the majority of the tests was mostly just > time-consuming.
Speaking of "pythonic" way, you should really consider using console_scripts entry point and proper python package, instead of using hacks for importing file directly. See here: https://setuptools.readthedocs.io/en/latest/setuptools.html#basic-use https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation https://github.com/QubesOS/qubes-core-agent-linux/blob/master/setup.py At this stage the change is still simple: 1. Rename the actual script to qvm_file_trust.py and put in to some subdirectory (python package name) - like qubesfiletrust, add empty __init__.py file there too. 2. Add setup.py with appropriate console_scripts entry point (pointing at `qubesfiletrust.qvm_file_trust:main`). 3. Move tests to that package too 4. Change hacky import into simply `import .qvm_file_trust` or `import qubesfiletrust.qvm_file_trust` In Makefile you can install it with: python setup.py install --root $(DESTDIR) And to run tests directly from git without installation, add "." (or full path to the source dir) to PYTHONPATH. > I also ran a few passes of [pylint](https://pypi.python.org/pypi/pylint) > over `qvm-file-trust` to really make sure it was clean and following > python's syntactical standards. A hint: take a look at pylint + unit tests integration with (Travis) CI we have here: https://github.com/qubesos/qubes-core-admin/ I'm not saying to do it now, but you may be interested in some examples. > I still plan to add a few more tests for the cli tool to the suite, as > well as those for the upcoming untrusted folder watching daemon. > > Speaking of which... > > ## Look out! It's a daemon! > > I took a little time this week to start work on the C daemon that will > watch all untrusted folders and mark any files created inside them as > untrusted as well. For starters, I've switched to making it a C++ deamon > instead. As far as I can tell there isn't any noticable performance > difference between the two languages, and C++ comes with a few > abstractions for various low-level things like file handling, meaning > less room for errors and bugs. I'm also much more experienced in C++, so > it seemed to the correct choice. +1 > While following the [Qubes Code Style > Guidelines](https://www.qubes-os.org/doc/coding-style/), I was able to > implement parsing of the global and local rule lists (including > functionality such as comment ignoring and override rules) as well as > being able to call on `qvm-file-trust` and set any file as untrusted > through it. I'm currently working on iterating through the rule list and > setting up an `inotify` watch on each folder (and its subfolders) for > any new files placed within them. > > In a very scientific test of setting up a basic C++ program to watch > `/tmp` and holding down tab after `ls /tmp/`, generating many > simulataneous inotify events, the CPU usage of the program was 0%. > Whether it will spike up when creating thousands of files within > untrusted folders remains to be seen, but I'm trying to keep the code as > efficient as possible. Slightly more scientific way would be something like this with output redirected to /dev/null, otherwise most of the time you're waiting for terminal output... For example: for x in `seq 100000`; do ls /tmp >/dev/null; done > One thing I wasn't entirely sure about was where to send logs. I'm not > sure if I should just log to syslog (and thus journald), or > /var/log/qubes/... (or both). It's simple enough to do either one. Either stderr (being redirected to syslog or file by the caller), or syslog. Leave log file management (including rotation, adding timestamps etc) to appropriate tools. Those places we still use /var/log/qubes directly are artifacts of the past mistakes... > ## Water-based File Managers > > I've started with modifying some of Dolphin's source code. Unfortunately > I've been having a little bit of trouble with their build tool, > [kdesrc-build](https://kdesrc-build.kde.org/). The amount of source code > necessary to build Dolphin is apparently quite large. Every time I try > to download the full dependencies of Dolphin it seems to stall halfway > through. > > This may not be the only way to work on Dolphin, but I haven't seen an > alternative so far... > > The other matter is that Whonix's current Dolphin version is 4.14.2, > while an updated Arch system's Dolphin version is 17.04.2 (and source is > 17.04.70). > > According to [this bug report](https://phabricator.whonix.org/T633), > Whonix should be using KDE Frameworks 5 in v14 (next release), which > should be included in Qubes soon following release. Due to this, I'm > wondering whether I should base my patches on top of Dolphin's latest > code or the legacy v4.14. If the latest, then it may be best to develop > on Arch instead of Qubes, or maybe I could find a pre-release build of > Whonix 14... I'd focus on the new version. Patrick, can you confirm the above? What is the base Debian version for Whonix 14? > For Nautilus I've just been using the latest code and it's going well. > Update on that in a bit! > > In any case, I've outlined the required changes that need to be made to > each file manager below: > > Nautilus: > > 1. Expose file open events to extensions > > 2. Respect not opening file if extension requests not to (upstream may > not like this part :) I'd not assume this - instead ask on appropriate mailing list. IMO this should also be acceptable if using clean mechanism for this. > Dolphin: > > 1. Work around the complete prevention of attempting to open files for > which we have no read permissions for (instead check with our cli tool > and decide what to do based on its output) > > 2. If the file is deemed an untrusted file, attempt to open it with our > dvm-desktop handler > > 3. Failing that, display an error > > That should about cover the functionality. Hopefully it'll be at least > partially upstream-able. I'm open to suggestions if anyone has any :) > > That about wraps it up for this week! As always the code is available > [here](https://github.com/anoadragon453/qubes-mime-types). And as usual, see comments there. - -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJZZKCFAAoJENuP0xzK19csSJYIAJQZ4DLTrtDdYLLaQOyqBV2e 9uTr0RMTYJjsMdbt/fC20xhvgdsJlKU8/8MHPOa8dnrt/nSyQLxyxdnJFtkHxgTh mEoLcb9vEzInaPIBUU5rMnMu0S5vDk0vI0r5pKFLKYMUgh0pC9GuFC2Z3q7esanI aNT0omcdzzgntszqGYSkjoOCjzo/hk4CST12EV/cTA/NajkPkulbuFoAkmslWo3Y KkED5kBnzxJXYuQdTfiz8fLT+7wqdQxxSEPRHXRR0Tz1F/qAySuRkT5high5rpb8 wL7t9p+TlmmiM4yFt9by9+HMb2BThj3l8VTQaFS9yiQlRec9Eod6YYOjAQ/QJiE= =DXuy -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20170711095517.GZ1095%40mail-itl. For more options, visit https://groups.google.com/d/optout.
