One thing about variant is that if the struct you are trying to insert is larger then (void delegate()).sizeof it will allocate the wrapped type on the gc heap.
This is not a concern if you want to have class templates as they are on the heap anyways and have a fixed size.
