Show a way to read or write outside allocated memory with this, or to cause a segmentation fault, and that will require a change in @safe. You're looking for something else, data safety rather than memory safety. You want to disallow unions and anything that lets you emulate them.
Does this count? http://dpaste.dzfl.pl/96db07a5104e