-----BEGIN PGP SIGNED MESSAGE-----
On Wed, Aug 09, 2017 at 07:28:21AM -0700, Andrew Morgan wrote:
> On 08/08/2017 03:38 AM, Andrew Morgan wrote:
> > Another report with some long-awaited news, enjoy!
> > Blog post with screenshots:
> > https://blog.amorgan.xyz/gsoc-weekly-progress-report-8.html
> > Otherwise text-only is below (with screenshots attached):
> > ---
> > Hey there, welcome to another report! Since the last one, you may
> > remember that I had finished setting up the build environment for
> > Nautilus inside of GNOME Builder and was just getting started on
> > properly doing the patch for supporting file trust attributes. Well, I'm
> > happy to report that the patch is now finished! Screenshots and details
> > below.
> > ## Working Demo
> > As screen recording in dom0 still isn't really doable yet, you'll have
> > to settle for some screenshots of it in action :)
> > ![Double-clicking a trusted file opens
> > normally...](images/open-trusted.png)
> > *Double-clicking a trusted file opens normally...*
> > ![Everyone knows gedit is the ultimate text
> > editor.](images/opened-gedit.png)
> > *Everyone knows gedit is the ultimate text editor.*
> > ![Whereas an untrusted file summons a DispVM!](images/open-untrusted.png)
> > *Whereas an untrusted file summons a DispVM!*
> > ![Virus free, yes-sir-ee!](images/opened-dispvm.png)
> > *Virus free, yes-sir-ee!*
> > This short demo makes use of a majority of all the different components
> > that have been worked on throughout the project. The now-patched
> > Nautilus makes a call to the patched NautilusPython which asks our
> > `qvm_trust` python extension to open a file, which it gives a response
> > based on what our cli tool, `qvm-file-trust`, says about it, and if it
> > is determined an untrusted file, `open-file-trust-based` will open it
> > securely in a DisposableVM. The contents of the file were never parsed
> > on the local machine!
> > The low-level part of this process was already working, but it took the
> > patch to Nautilus allowing us to set off the chain from the GUI to
> > really bring it all together. Let's talk about how that was done and how
> > it works.
> > ## Nautilus and NautilusPython
> > By default, Nautilus does not support extensions written in Python, but
> > rather only those written in C. To get around this, a _Nautilus C
> > extension called NautilusPython_ was created. This extension has the
> > ability to provide a bridging interface from Nautilus' extension
> > infastructure to Python, handling any calls Python programs make to
> > Nautilus and calling them from C. Arguments and return variables are all
> > passed along seamlessly.
> > This is a really nice feature, as writing extensions in Python are about
> > 90% less work than writing them in C, due to the large amount of
> > boiler-plate associated with setting up and tearing down an extension.
> > If you recall, the point of patching Nautilus was to allow extensions to
> > be able to be notified of when a file is being opened, and block that
> > request if necessary. I didn't want to subject myself or anyone else to
> > only being able to write an extension that made use of this new
> > functionality in C, thus patching NautilusPython as well as Nautilus
> > itself to add our new `file_open` function was necessary. So how was
> > each one done then?
> > Patching NautilusPython to add a new method was relatively simple, you
> > only need look at the existing methods that an extension can subclass,
> > find one that is functionally similar your own (I used
> > `update_file_info` as my guide), then insert your own method
> > (`file_open` in this case) at all the relevant points as well as create
> > the appropriate function bodies and set argument/return variables
> > appropriately.
> > So overall NautilusPython wasn't too bad really. The real time sink was
> > Nautilus itself.
> > ## Patching Nautilus
> > First of all, I almost immediately ditched GNOME Builder after my last
> > blog post. It's not that it was a bad IDE, far from it actually, I quite
> > liked the interface and the various tools it contained. The ability to
> > download a project from git, package and run it inside a Flatpak, and
> > then send patches upstream all from within the editor was quite
> > appealing! The first major showstopper however, was that the version of
> > Nautilus that Qubes makes use of (v3.22.3), doesn't support Flatpak.
> > While building and running from GNOME Builder without Flatpak seemed
> > possible, no matter what I tried I could not get extensions to function
> > properly with the GNOME Builder version.
> > After a lot of fiddling about, I settled on what I probably should have
> > been using since the very beginning:
> > [qubes-builder](https://github.com/qubesos/qubes-builder)!
> > Not only does it build the correct version (and supports extensions),
> > but I knew that whatever I did with this build system would be exactly
> > what users would be using when they built Qubes' templates or got them
> > from the build servers. So, unfortunately no Flatpak'd builds, but I do
> > have pre-packaged RPMs with debug-symbols! That's probably even better,
> > right?
> > I'll post them in a separate blog post with instructions on how to set
> > everything up soon.
> > qubes-builder worked very well and was quite easy to drill into and mess
> > about with, as it is mostly just a collection of various scripts. I
> > eventually built a script that ran in dom0 that would build Nautilus in
> > my development VM, transfer it to my testing VM, delete the old version
> > and install the new version.
> > The only downside to this process was that building Nautilus took quite
> > a while each time (2-3m) which made quick changes quite an agonizing
> > process. I believe this was due to both having to rebuild _every_ file
> > each time (as they were packaged in a .tar.xz before being built,
> > meaning all files were considered to be modified) as well as a call to
> > `dnf` to fetch any updates was made, _every time_. I found out the
> > latter when working without internet and realizing my could would not
> > build! Each of these things probably could have been worked out or
> > changed if I had felt the need to, but I just stuck with the loading
> > times and usually had internet, so it was not too terrible.
> > Was _was_ terrible and time-consuming were a couple of very strange
> > issues I had during the creation of the patch.
> > The first was the dreaded _provider issue_. You see, every time Nautilus
> > calls a method that is meant to be subclassed by a Nautilus extension,
> > you must _provide a provider_. This can be an `INFO` provider, `MENU`
> > provider, and so on. Each method only accepts one of each type,
> > typically corresponding to the category of method you were calling. For
> > the `file_open` method I decided to stick it under the `INFO` provider
> > category, as that had to do with information about files, which I felt
> > 'when they were opened' best fell into.
> > Well, these providers don't just pop out of nowhere. You have to create
> > them! You can do that by calling the intuitively-named
> > `nautilus_module_get_extensions_for_type` method, providing it the
> > `NAUTILUS_TYPE_INFO_PROVIDER` enum. It is then _supposed_ to return a
> > provider with a type that matches the one you gave it, but no matter
> > what I did it would always just return a `NULL` pointer! I spent _days_
> > on this problem, trying loads of different things. Every other call of
> > `nautilus_module_get_extensions_for_type` had no special setup or
> > initialization, leading me confused why my own calling of it brought
> > such different results.
> > After picking through many lines of code, talking with GNOME people over
> > IRC who were currently GUADEC'ing, and trying to understand the whole
> > GNOME development ecosystem, I eventually traced it all back to
> > NautilusPython not finding an extension that made use of a subclassed
> > method from the `INFO` method type. But wait, we _do_ have an extension
> > that does that with `qvm_trust`, so why doesn't it work??
> > Well I hit my head after this one folks. The version of NautilusPython
> > that Nautilus was loading in my test VM was the original, from-repo
> > version, not our modified version. **D'oh!**
> > "Well that's just a simple fix, I just have to build it and import it
> > into Nautilus in the testing VM!", I assumed, before realizing that
> > while my modified NautilusPython built, Nautilus _would not accept it_.
> > Nautilus would just _hang_ with little to no output.
> > After a few emails, Marek pointed me in the correct direction with the
> > `nautilus-3.0` branch of NautilusPython, which, while 6 years since the
> > last commit, worked with the latest version. Alright no problem. Rebase
> > my commits on that branch, rebuild, transfer the newly-built
> > shared-library over and bam, Nautilus runs again! And we have a provider
> > that's not NULL, hooray!
> > And then, ladies and gentlemen, problem #2 reared _its_ ugly head.
> > Here's the code to get a list (`GList*`) of providers to use with a
> > Nautilus extension:
> > providers = nautilus_module_get_extensions_for_type
> > (NAUTILUS_TYPE_INFO_PROVIDER);
> > While we were getting a _non-null_ provider value, the value we received
> > was _different_ from the one returned by the method we called. "Wat?"
> > you say, and I too said the same. For some reason we were returning a
> > 64-bit `GList` pointer from `nautilus_module_get_extensions_for_type`,
> > but storing a 32-bit `GList` pointer in `providers`.
> > I tried all sorts of weird tricks to get this fixed. Nobody I talked to
> > seemed to have a solution, and after wrestling with it for a few days I
> > begrudingly settled on this very elegant workaround:
> > // TODO: Providers ends up as 0x55f65ac0
> > // when method is returning 0x555555f65ac0, very odd...
> > providers = (GList*) (nautilus_module_get_extensions_for_type
> > (NAUTILUS_TYPE_INFO_PROVIDER) + 0x555500000000);
> > Yes I know this is terrible. It did work though, and I needed to stop
> > hitting this strange bug and get on with patching, so that's where it
> > currently sits. I'm still actively looking for a proper solution. If
> > anyone knows why a 64bit pointer might get truncated to a 32bit pointer
> > after a function call in C please do let me know! You can find the full
> > code
> > [here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-mime-actions.c#L2496)
> > and the `type` method
> > [here](https://github.com/anoadragon453/nautilus/blob/master/src/nautilus-module.c#L269).
> > After this was out of the way, and a handful of more methods were
> > implemented, we finally had a working solution between the `qvm_trust`
> > extension and our patched Nautilus! It was beautiful, really.
> > ## Conclusion and going forward
> > My [email](https://amorgan.xyz/upstream-request.html) to the [Nautilus
> > mailing list](https://mail.gnome.org/archives/nautilus-list/) is still
> > awaiting moderation. GUADEC has now concluded, and with no update on the
> > situation it seems I need to do some more active poking of people to get
> > moderation approval. We'll see where that lands, the code still needs to
> > be cleaned up a bit before sending it upstream anyways, so I'm not too
> > worried.
> > At this point in the project, we essentially have the following tasks left:
> > * Patch KDE Dolphin to provide the same functionality as Nautilus now does
> > * Finish up some small tasks with the daemon like logging and some more
> > interfacing with the cli tool
> > * Code cleanup, linting, tests
> > * Integrate into main QubesOS repos
> > I'm going to prioritize the latter 3 bullet points at this point to
> > ensure we get a really solid core implementation down of the project. I
> > don't want to spend another couple weeks patching Dolphin only to run
> > out of time for a general cleanup and other polish. I will try to pour
> > through the Dolphin code and note down everything that one would need to
> > do with Dolphin to make it work with file trust attributes, so that in
> > the future either myself or someone else can give it a shot.
> > It may end up that Dolphin already has most of the functionality and the
> > patch will be very small, we'll see!
> > Anyways, for now I'm going to start on finishing the remaining small
> > tasks, and will post any questions regarding packaging and/or cleanup to
> > the qubes-devel mailing list.
> > As always, you can find the code
> > [here](https://github.com/anoadragon453/qubes-mime-types).
> > And as of now, you can also find the patched code for
> > [Nautilus](https://github.com/anoadragon453/nautilus) and
> > [NautilusPython](https://github.com/anoadragon453/nautilus-python/tree/nautilus-3.0)
> > on Github as well. Make sure that when you're inspecting the
> > NautilusPython code, you're looking at the `nautilus-3.0` branch. That's
> > where all the important commits are!
> > That's all for now, thanks for reading!
> The git commits for nautilus/python seem like a bit of a mess at the
> moment. Are you currently reviewing them individually or can I squash
> them into just a few and force push to the repo?
Feel free to squash them.
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
-----END PGP SIGNATURE-----
You received this message because you are subscribed to the Google Groups
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send email to firstname.lastname@example.org.
To view this discussion on the web visit
For more options, visit https://groups.google.com/d/optout.