-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Wed, Aug 09, 2017 at 07:28:21AM -0700, Andrew Morgan wrote: > On 08/08/2017 03:38 AM, Andrew Morgan wrote: > > Another report with some long-awaited news, enjoy! > > > > Blog post with screenshots: > > https://blog.amorgan.xyz/gsoc-weekly-progress-report-8.html > > > > Otherwise text-only is below (with screenshots attached): > > > > --- > > > > Hey there, welcome to another report! Since the last one, you may > > remember that I had finished setting up the build environment for > > Nautilus inside of GNOME Builder and was just getting started on > > properly doing the patch for supporting file trust attributes. Well, I'm > > happy to report that the patch is now finished! Screenshots and details > > below. > > > > ## Working Demo > > > > As screen recording in dom0 still isn't really doable yet, you'll have > > to settle for some screenshots of it in action :) > > > >  > > *Double-clicking a trusted file opens normally...* > > > >  > > *Everyone knows gedit is the ultimate text editor.* > > > >  > > *Whereas an untrusted file summons a DispVM!* > > > >  > > *Virus free, yes-sir-ee!* > > > > This short demo makes use of a majority of all the different components > > that have been worked on throughout the project. The now-patched > > Nautilus makes a call to the patched NautilusPython which asks our > > `qvm_trust` python extension to open a file, which it gives a response > > based on what our cli tool, `qvm-file-trust`, says about it, and if it > > is determined an untrusted file, `open-file-trust-based` will open it > > securely in a DisposableVM. The contents of the file were never parsed > > on the local machine! > > > > The low-level part of this process was already working, but it took the > > patch to Nautilus allowing us to set off the chain from the GUI to > > really bring it all together. Let's talk about how that was done and how > > it works. > > > > ## Nautilus and NautilusPython > > > > By default, Nautilus does not support extensions written in Python, but > > rather only those written in C. To get around this, a _Nautilus C > > extension called NautilusPython_ was created. This extension has the > > ability to provide a bridging interface from Nautilus' extension > > infastructure to Python, handling any calls Python programs make to > > Nautilus and calling them from C. Arguments and return variables are all > > passed along seamlessly. > > > > This is a really nice feature, as writing extensions in Python are about > > 90% less work than writing them in C, due to the large amount of > > boiler-plate associated with setting up and tearing down an extension. > > If you recall, the point of patching Nautilus was to allow extensions to > > be able to be notified of when a file is being opened, and block that > > request if necessary. I didn't want to subject myself or anyone else to > > only being able to write an extension that made use of this new > > functionality in C, thus patching NautilusPython as well as Nautilus > > itself to add our new `file_open` function was necessary. So how was > > each one done then? > > > > Patching NautilusPython to add a new method was relatively simple, you > > only need look at the existing methods that an extension can subclass, > > find one that is functionally similar your own (I used > > `update_file_info` as my guide), then insert your own method > > (`file_open` in this case) at all the relevant points as well as create > > the appropriate function bodies and set argument/return variables > > appropriately. > > > > So overall NautilusPython wasn't too bad really. The real time sink was > > Nautilus itself. > > > > ## Patching Nautilus > > > > First of all, I almost immediately ditched GNOME Builder after my last > > blog post. It's not that it was a bad IDE, far from it actually, I quite > > liked the interface and the various tools it contained. The ability to > > download a project from git, package and run it inside a Flatpak, and > > then send patches upstream all from within the editor was quite > > appealing! The first major showstopper however, was that the version of > > Nautilus that Qubes makes use of (v3.22.3), doesn't support Flatpak. > > While building and running from GNOME Builder without Flatpak seemed > > possible, no matter what I tried I could not get extensions to function > > properly with the GNOME Builder version. > > > > After a lot of fiddling about, I settled on what I probably should have > > been using since the very beginning: > > [qubes-builder](https://github.com/qubesos/qubes-builder)! > > > > Not only does it build the correct version (and supports extensions), > > but I knew that whatever I did with this build system would be exactly > > what users would be using when they built Qubes' templates or got them > > from the build servers. So, unfortunately no Flatpak'd builds, but I do > > have pre-packaged RPMs with debug-symbols! That's probably even better, > > right? > > > > I'll post them in a separate blog post with instructions on how to set > > everything up soon. > > > > qubes-builder worked very well and was quite easy to drill into and mess > > about with, as it is mostly just a collection of various scripts. I > > eventually built a script that ran in dom0 that would build Nautilus in > > my development VM, transfer it to my testing VM, delete the old version > > and install the new version. > > > > The only downside to this process was that building Nautilus took quite > > a while each time (2-3m) which made quick changes quite an agonizing > > process. I believe this was due to both having to rebuild _every_ file > > each time (as they were packaged in a .tar.xz before being built, > > meaning all files were considered to be modified) as well as a call to > > `dnf` to fetch any updates was made, _every time_. I found out the > > latter when working without internet and realizing my could would not > > build! Each of these things probably could have been worked out or > > changed if I had felt the need to, but I just stuck with the loading > > times and usually had internet, so it was not too terrible. > > > > Was _was_ terrible and time-consuming were a couple of very strange > > issues I had during the creation of the patch. > > > > The first was the dreaded _provider issue_. You see, every time Nautilus > > calls a method that is meant to be subclassed by a Nautilus extension, > > you must _provide a provider_. This can be an `INFO` provider, `MENU` > > provider, and so on. Each method only accepts one of each type, > > typically corresponding to the category of method you were calling. For > > the `file_open` method I decided to stick it under the `INFO` provider > > category, as that had to do with information about files, which I felt > > 'when they were opened' best fell into. > > > > Well, these providers don't just pop out of nowhere. You have to create > > them! You can do that by calling the intuitively-named > > `nautilus_module_get_extensions_for_type` method, providing it the > > `NAUTILUS_TYPE_INFO_PROVIDER` enum. It is then _supposed_ to return a > > provider with a type that matches the one you gave it, but no matter > > what I did it would always just return a `NULL` pointer! I spent _days_ > > on this problem, trying loads of different things. Every other call of > > `nautilus_module_get_extensions_for_type` had no special setup or > > initialization, leading me confused why my own calling of it brought > > such different results. > > > > After picking through many lines of code, talking with GNOME people over > > IRC who were currently GUADEC'ing, and trying to understand the whole > > GNOME development ecosystem, I eventually traced it all back to > > NautilusPython not finding an extension that made use of a subclassed > > method from the `INFO` method type. But wait, we _do_ have an extension > > that does that with `qvm_trust`, so why doesn't it work?? > > > > Well I hit my head after this one folks. The version of NautilusPython > > that Nautilus was loading in my test VM was the original, from-repo > > version, not our modified version. **D'oh!** > > > > "Well that's just a simple fix, I just have to build it and import it > > into Nautilus in the testing VM!", I assumed, before realizing that > > while my modified NautilusPython built, Nautilus _would not accept it_. > > Nautilus would just _hang_ with little to no output. > > > > After a few emails, Marek pointed me in the correct direction with the > > `nautilus-3.0` branch of NautilusPython, which, while 6 years since the > > last commit, worked with the latest version. Alright no problem. Rebase > > my commits on that branch, rebuild, transfer the newly-built > > shared-library over and bam, Nautilus runs again! And we have a provider > > that's not NULL, hooray! > > > > And then, ladies and gentlemen, problem #2 reared _its_ ugly head. > > > > Here's the code to get a list (`GList*`) of providers to use with a > > Nautilus extension: > > > > providers = nautilus_module_get_extensions_for_type > > (NAUTILUS_TYPE_INFO_PROVIDER); > > > > While we were getting a _non-null_ provider value, the value we received > > was _different_ from the one returned by the method we called. "Wat?" > > you say, and I too said the same. For some reason we were returning a > > 64-bit `GList` pointer from `nautilus_module_get_extensions_for_type`, > > but storing a 32-bit `GList` pointer in `providers`. > > > > I tried all sorts of weird tricks to get this fixed. Nobody I talked to > > seemed to have a solution, and after wrestling with it for a few days I > > begrudingly settled on this very elegant workaround: > > > > // TODO: Providers ends up as 0x55f65ac0 > > // when method is returning 0x555555f65ac0, very odd... > > providers = (GList*) (nautilus_module_get_extensions_for_type > > (NAUTILUS_TYPE_INFO_PROVIDER) + 0x555500000000); > > > > Yes I know this is terrible. It did work though, and I needed to stop > > hitting this strange bug and get on with patching, so that's where it > > currently sits. I'm still actively looking for a proper solution. If > > anyone knows why a 64bit pointer might get truncated to a 32bit pointer > > after a function call in C please do let me know! You can find the full > > code > > [here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-mime-actions.c#L2496) > > and the `type` method > > [here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-module.c#L269). > > > > After this was out of the way, and a handful of more methods were > > implemented, we finally had a working solution between the `qvm_trust` > > extension and our patched Nautilus! It was beautiful, really. > > > > ## Conclusion and going forward > > > > My [email](https://amorgan.xyz/upstream-request.html) to the [Nautilus > > mailing list](https://mail.gnome.org/archives/nautilus-list/) is still > > awaiting moderation. GUADEC has now concluded, and with no update on the > > situation it seems I need to do some more active poking of people to get > > moderation approval. We'll see where that lands, the code still needs to > > be cleaned up a bit before sending it upstream anyways, so I'm not too > > worried. > > > > At this point in the project, we essentially have the following tasks left: > > > > * Patch KDE Dolphin to provide the same functionality as Nautilus now does > > * Finish up some small tasks with the daemon like logging and some more > > interfacing with the cli tool > > * Code cleanup, linting, tests > > * Integrate into main QubesOS repos > > > > I'm going to prioritize the latter 3 bullet points at this point to > > ensure we get a really solid core implementation down of the project. I > > don't want to spend another couple weeks patching Dolphin only to run > > out of time for a general cleanup and other polish. I will try to pour > > through the Dolphin code and note down everything that one would need to > > do with Dolphin to make it work with file trust attributes, so that in > > the future either myself or someone else can give it a shot. > > > > It may end up that Dolphin already has most of the functionality and the > > patch will be very small, we'll see! > > > > Anyways, for now I'm going to start on finishing the remaining small > > tasks, and will post any questions regarding packaging and/or cleanup to > > the qubes-devel mailing list. > > > > As always, you can find the code > > [here](https://github.com/anoadragon453/qubes-mime-types). > > > > And as of now, you can also find the patched code for > > [Nautilus](https://github.com/anoadragon453/nautilus) and > > [NautilusPython](https://github.com/anoadragon453/nautilus-python/tree/nautilus-3.0) > > on Github as well. Make sure that when you're inspecting the > > NautilusPython code, you're looking at the `nautilus-3.0` branch. That's > > where all the important commits are! > > > > That's all for now, thanks for reading! > > > > Marek, > > The git commits for nautilus/python seem like a bit of a mess at the > moment. Are you currently reviewing them individually or can I squash > them into just a few and force push to the repo?
Feel free to squash them. - -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJZiyRaAAoJENuP0xzK19csug0H+wRi9RLWQWHmMBznsWUFquvL qF4UOKrUpTd463fUGdzOIIWqlox9gciu7X3mOIsXBmoCTyt9ouq2dsHV80WSpku/ 9Wn/smKf01jQNYgRq9ZJ80jOU3TjAZcTwMgG1KynHPTKOJHf4EBQmCsKIjZtyAVk DbnkHIFxb0npUIFqUqr7qJF/f9Sx0WgTR3LlHrbmIIkXHPr01K9YN/SGqIJrw8Wh JWV6m0mJHjQXSnJccG58FbLBdA0J/Mf0T20mV+5vQKj1QwH0bOci6+V7PMC9HKN2 xEGPR8dF14gQvwEq1EHWN8kdkjD00ttM1eTdzz4CAUzPyLGsAppP9lGwTsd2B84= =92f+ -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20170809150354.GM1145%40mail-itl. For more options, visit https://groups.google.com/d/optout.
