On 07/11/2017 04:14 PM, Marek Marczykowski-Górecki wrote: > On Tue, Jul 11, 2017 at 04:09:33PM -0700, Andrew Morgan wrote: >> On 07/11/2017 04:03 PM, Marek Marczykowski-Górecki wrote: >>> On Tue, Jul 11, 2017 at 03:39:57PM -0700, Andrew Morgan wrote: >>>> On 07/11/2017 02:55 AM, Marek Marczykowski-Górecki wrote: >>>>> On Mon, Jul 10, 2017 at 09:38:22PM -0700, Andrew Morgan wrote: >>>>>> I also ran a few passes of [pylint](https://pypi.python.org/pypi/pylint) >>>>>> over `qvm-file-trust` to really make sure it was clean and following >>>>>> python's syntactical standards. >>>>> >>>>> A hint: take a look at pylint + unit tests integration with (Travis) CI >>>>> we have here: >>>>> https://github.com/qubesos/qubes-core-admin/ >>>>> >>>>> I'm not saying to do it now, but you may be interested in some examples >>> >>>> Yeah I wasn't sure if it would be necessary to set up travis on my repo >>>> since the end goal is to integrate it into the main repos at some point. >>> >>> Well, "set up travis" is quite simple - add .travis.yml (look at >>> existing repositories), then login at travis-ci.org and enable >>> repository. Anyway, I don't think it's necessary right now. >>> >>>> I'm pretty sure unittest has a pylint plugin I could utilize locally >>>> (and thus before every commit). >>> >>>>> Slightly more scientific way would be something like this with output >>>>> redirected to /dev/null, otherwise most of the time you're waiting for >>>>> terminal output... For example: >>>>> >>>>> for x in `seq 100000`; do ls /tmp >/dev/null; done >>> >>>> Unfortunately only scanning the directory with `ls /tmp<tab>` actually >>>> produces any events, just ls'ing it doesn't. So instead I ran the >>>> following, which should more accurately cover our use case (notified of >>>> files being created in a watched directory): >>> >>>> for x in `seq 100000`; do touch /tmp/asd; rm /tmp/asd; done >>> >>>> Our C++ program seemed to max out at 0.7% CPU, which I think is pretty >>>> good :) >>> >>>> One thing that may take more CPU is when we call our python program to >>>> mark each of these new files as untrusted though. It might not end well >>>> if we have to start up and shut down the python interpreter for every >>>> single new file... >>> >>>> Possible solutions might be: >>> >>>> 1. Mark the file as untrusted manually (chmod 0, xattr) through the C++ >>>> program. (Con: if we change the definition of an untrusted file down the >>>> line then we have to edit the C++ program as well) >>> >>>> 2. Every second or so, grab all newly created files and just do a batch >>>> run of `qvm-file-trust` with all of them. (Con: still need to start the >>>> python interpreter but at least we're not hammering it now) >>> >>> For now I'd ignore the problem. This happens when new untrusted file is >>> created, which should be rare case assuming proper compartmentalization. >>> But if it will be an issue, I'd go with 2, to not duplicate definition >>> of untrusted file (bugs like this could be fatal). >>> >>> > >> A good example someone brought up a while ago was starting a build >> inside of an untrusted folder. Many, many binary files will then be >> created all at once. > > That would be very bad idea: > 1. In essence it means executing those untrusted files locally (think: > Makefile) > 2. We chmod 0 untrusted files, so it wouldn't work > > Instead, DispVM should be used for this. > >> I'll do some benchmarks once everything is hooked up to decide what the >> best course of action will be. > > +1 > >
Hey Marek, Just wanted to quickly point out that the next report is coming out soon. Just wanted to finish up the C++ daemon beforehand. It can already track untrusted folders and their subfolders, as well as mark any new folders or files that are created or moved inside of them. I'm currently trying to work out a bug where inotify_watch calls will fail around the 8000th folder that's created or moved in. I'm assuming this probably has to do with a limit coded somewhere so I'm looking out for that. I also packaged qvm-file-trust as a python module. Installation goes great, but I get some errors after trying to run the installed script: https://gist.github.com/anonymous/c51749ce9f951e4b53fd4da9ed5a925b qvm_file_trust.py is placed in /rw/usrlocal/bin/qvm_file_trust.py qvm-file-trust is placed in /usr/local/qvm-file-trust but is unable to find the correct module Just moving qvm-file-trust to /rw/usrlocal/bin/ makes it work, so seems like it's looking relatively for files... Also did a test with moving in an enormous folder, daemon took up 16% CPU for a second in htop then right back to 0%, so seems pretty well optimized for now. inotify finds all the files and folders in way until a few hundred milli-seconds, so we may need to scale our period for calling qvm-file-trust with a list of files down a bit (unless python can take in 10K+ full filepaths as arguments). Thanks Andrew Morgan -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/okpfsb%24rnl%241%40blaine.gmane.org. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: OpenPGP digital signature
