rpc.svcgssd is also needed on clients in order to support NFSv4.0 callbacks. It was moved from nfs-kernel-server to nfs-common for this reason. See Debian bug #651558.
Apparently the task of starting rpc.svcgssd under SysV init is still entrusted to the nfs-kernel-server package. Maybe something needs to be done about that. If you are using systemd, you may find the file systemd/README in the source package to be of interest. The relevant portion reads: "rpc.gssd and rpc.svcgssd are assumed to be needed if /etc/krb5.keytab is present. If a site needs this file present but does not want the gss daemons running, it should create /etc/systemd/system/rpc-gssd.service.d/01-disable.conf and /etc/systemd/system/rpc-svcgssd.service.d/01-disable.conf containing [Unit] ConditionNull=false " I think this (or equivalent information; I'd have suggested "systemctl disable" instead of the above approach) should be included somewhere under /usr/share/doc/nfs-common/. As for the side question on how to disable version 4.0 but not 4.1: try passing --no-nfs-version=4.0 to nfsd. (I haven't tested this myself yet, only read utils/nfsd/nfsd.c. The man page is too terse about this.)

