Just a quick note for publickey-only sshd users that if you upgrade to openssh 5.4 the AuthorizedKeysFile entry in sshd_config may need to be updated or else you won't be able to login to your system.
If you have: AuthorizedKeysFile .ssh/authorized_keys (which is the default example in the config file) change it to this: AuthorizedKeysFile %h/.ssh/authorized_keys and then restart sshd. Thanks to Doktor Notor who posted the solution on bugzilla. http://bugs.gentoo.org/show_bug.cgi?id=308939
