I got it working. I have a pretty hefty amount of vether0 and vether0:network in my pf.conf that I changed to vport0 and vport0:network.
That fixed every single thing! I somehow completely forgot about all the vether0 pf rules which isolates the the various local systems so VMs are isolated from being able to do anything malicious to any local systems. I silently redirect the VMs' dns and ntp calls to my OpenBSD services to harden them a bit too. -Luke