Hi, We're having some trouble on a linux ssh shell server. There are currently 300 or so account, and when any sizable number of these connect simultaneously, subsequent connections hang after authentication for any interactive session.
Running non-interactive commands (say, `ls`) works fine, but running anything interactive (tried bash and zsh in this case) hangs the connection. When the connection is hanged, if the client is in verbose mode, I see keepalive packets being transmitted regularly, so the connection is there... Just not doing anything. This has been reported for many different clients, including putty on win32 and versions of openssh from 4.2 to 4.5 on linux, bsd and osx. The server itself has "OpenSSH_4.2p1, OpenSSL 0.9.7l 28 Sep 2006" installed. Has anyone encountered this before? We suspect we're running into some kind of resource limit, but ssh not explicitely failing or logging anything unusual is making this hard to debug. Follows a transcript of an openssh 4.2 trying to connect and get a shell: % ssh -vv anapnea.net OpenSSH_4.2p1, OpenSSL 0.9.7l 28 Sep 2006 debug1: Reading configuration data /etc/ssh_config debug2: ssh_connect: needpriv 0 debug1: Connecting to anapnea.net [64.79.210.115] port 22. debug1: Connection established. debug1: identity file /Users/dave/.ssh/identity type -1 debug2: key_type_from_name: unknown key type '-----BEGIN' debug2: key_type_from_name: unknown key type 'Proc-Type:' debug2: key_type_from_name: unknown key type 'DEK-Info:' debug2: key_type_from_name: unknown key type '-----END' debug1: identity file /Users/dave/.ssh/id_rsa type 1 debug1: identity file /Users/dave/.ssh/id_dsa type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_4.5 debug1: match: OpenSSH_4.5 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.2 debug2: fd 3 setting O_NONBLOCK debug1: Miscellaneous failure No credentials cache found debug1: Miscellaneous failure No credentials cache found debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,[EMAIL PROTECTED],aes128-ctr,aes192-ctr,aes256-ctr debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,[EMAIL PROTECTED],aes128-ctr,aes192-ctr,aes256-ctr debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,[EMAIL PROTECTED],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,[EMAIL PROTECTED],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: [EMAIL PROTECTED],zlib,none debug2: kex_parse_kexinit: [EMAIL PROTECTED],zlib,none debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,[EMAIL PROTECTED],aes128-ctr,aes192-ctr,aes256-ctr debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,[EMAIL PROTECTED],aes128-ctr,aes192-ctr,aes256-ctr debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,[EMAIL PROTECTED],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,[EMAIL PROTECTED],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,[EMAIL PROTECTED] debug2: kex_parse_kexinit: none,[EMAIL PROTECTED] debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_init: found hmac-md5 debug1: kex: server->client aes128-cbc hmac-md5 [EMAIL PROTECTED] debug2: mac_init: found hmac-md5 debug1: kex: client->server aes128-cbc hmac-md5 [EMAIL PROTECTED] debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug2: dh_gen_key: priv key bits set: 115/256 debug2: bits set: 511/1024 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'anapnea.net' is known and matches the RSA host key. debug1: Found key in /Users/dave/.ssh/known_hosts:57 debug2: bits set: 526/1024 debug1: ssh_rsa_verify: signature correct debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /Users/dave/.ssh/id_rsa (0x3049f6) debug2: key: /Users/dave/.ssh/identity (0x0) debug2: key: /Users/dave/.ssh/id_dsa (0x0) debug1: Authentications that can continue: publickey,keyboard-interactive debug1: Next authentication method: publickey debug1: Offering public key: /Users/dave/.ssh/id_rsa debug2: we sent a publickey packet, wait for reply debug1: Authentications that can continue: publickey,keyboard-interactive debug1: Trying private key: /Users/dave/.ssh/identity debug1: Trying private key: /Users/dave/.ssh/id_dsa debug2: we did not send a packet, disable method debug1: Next authentication method: keyboard-interactive debug2: userauth_kbdint debug2: we sent a keyboard-interactive packet, wait for reply debug2: input_userauth_info_req debug2: input_userauth_info_req: num_prompts 1 Password: debug2: input_userauth_info_req debug2: input_userauth_info_req: num_prompts 0 debug1: Enabling compression at level 6. debug1: Authentication succeeded (keyboard-interactive). debug1: channel 0: new [client-session] debug2: channel 0: send open debug1: Entering interactive session. debug2: callback start debug2: client_session2_setup: id 0 debug2: channel 0: request pty-req confirm 0 debug2: channel 0: request shell confirm 0 debug2: fd 3 setting TCP_NODELAY debug2: callback done debug2: channel 0: open confirm rwindow 0 rmax 32768 debug2: channel 0: rcvd adjust 131072 [Hangs here without opening a shell... After that, periodically, a keepalive tacks on] debug1: client_input_channel_req: channel 0 rtype [EMAIL PROTECTED] reply 1 debug1: client_input_channel_req: channel 0 rtype [EMAIL PROTECTED] reply 1 debug1: client_input_channel_req: channel 0 rtype [EMAIL PROTECTED] reply 1 Thanks in advance for any insight you could provide here. - Dave
