Quack,I also hit this problem recently on Buster. I also do have specific Listen directives on this machine but all my IPs (4 and 6) are statically defined in /etc/network/interfaces.
I looked at the systemd service file and I believe that After=network.target is not enough.
According to upstream documentation: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ I should be replaced by: After=network-online.target Wants=network-online.target Regards. \_o< -- Marc Dequènes