I'm hearing some complicated attack vectors being tossed around in here (password sniffing, mitm, etc) -- don't forget about a pretty simple one: dictionary attacks. I recently took over a project for a small-medium sized client and upon looking at the secure log noticed 50k+/day dictionary attacks against SSH. I installed fail2ban and now get 5-6 emails daily about brute-force hack attempts.
YMMV, but food for thought. -D
