I knew someone would ask. This is how I pictured it at that moment:
If server intercepts the encrypted password from one client and sends its own
instead of legitimate one you could simply implement manual verification.
Jim tells Bob: "Ok bob I'm gonna give you remote support. Encrypted password
is doughnut. Start up the program"
Bob: "Ok" ./ReMoteSupport doughnut.
ReMoteSupport sends public key expecting that response will be string
doughnut encrypted with his public key. If its not Bob sues Chris :)