Ok first simple test :
a non-encrypted drive in ext4 which I can't safely remove (With Nautilus) on
my install (not authorized to perform operation).
Yet on a normal Trisquel install, while I have to force the action since it
says the drive is busy, it just works.
The disk ID is a weird format like 0x00randomchars (different from the
internal disk and it's LUKS partitions),
whereas my user ID is four numbers.
But you know what, I just tested it on a Debian netinstall, it works also.
It's probably a faulty install, it's probably easier to start over. Plus it's
a clue that it's not a HDD issue (gotta test further though).