Hi Stas thanks for the feedback. I've added more info for more clarity. It's absolutely impossible to treat notices as errors in PHP, so I assume everybody thinks the same. If someone converts notices to ErrorExceptions or something, it's their fault. A notice in tests is exactly what a deprecation is supposed to do, force people to update their code.
Regarding the notice level, it's a bit confusing picking one. E_DEPRECATED, E_STRICT, E_NOTICE... I'm not sure what it should be.