It turned out to be a bug in VxVM 4.0 and has been fixed in 4.0 MP2:
The 4.0MP2 release notes (http://support.veritas.com/docs/277626) has
the following entry under the fixed incidences section:
311488 The entries in /dev/vx/[r]dsk/ are not cleared during a
reboot/system start-up after
Does anyone know why there is a pause in i/o when using the adaptive
iopolicy if you block one path? We have found adaptive to work the best
in our environment for performance and for the time-to-fail also but
unsure why when we test by blocking one path on the SAN that there is a
pause of about