Having looked at the code for the MANIFEST check, I see that it uses File::Find to enumerate what's present, and then checks that each item in the MANIFEST is in the set of files that it found. So the problem seems to amount to File::Find failing to recurse into subdirectories. I suspect that the installation is happening on a filesystem that shows nlink==2 for all directories, thus misleading File::Find into applying the no-subdirectories optimisation.
I think the second run, with sudo, went wrong because it was working from an unclean build directory. Only the first failure is meaningful. -zefram