As a bonus reply to some of my own questions, I found this out
https://tracker.debian.org/pkg/caja-seahorse
Installing it was easy and gave me the ability to import signatures, verify
it, etc. All from a simple right click and from within caja. I still have not
found a solution for secure deleting files, I am still using the CLI for
that. But hope to find something. Any help greatly appreciated.
P.S.: Yes I am aware Magic Banana explained how to use nautilus and
gnome-shell. But I am trying to stick with the default mate desktop in which
I have already invested some time configuring things my way. So, yeah, I am
sticking with caja.