Hi Dan and all, After a few hours of reading up on various sources I think I finally managed to wrap my head around this. This will be a long post with a lot of details. However, I'll summarize the important conclusions from the end of this message in the very beginning, too:
The bottom line is this: 1.) Current Linux kernel on RHEL 5 and 6 clones are vulnerable to a privilege escalation performed by a local user (or process). Updated kernels should hit the official vendor repositories soon. ETA unknown, though. 2.) SSHd Spam Exploit (libkeyutils.so.1.9): Aside from that there is currently an ongoing attack which started sometime in December 2012 and which seems to primarily target boxes with control panels. This attack modifies OpenSSH's behavior via a manually supplemented libkeyutils.so.1.9 library, which allows the attacker to login via SSH and to perform commands on the trojaned servers. It appears that this trojan also sends your "root" password to a certain collector box and that hacked servers are mostly used to send SPAM. The entry vector how this infection is/was carried out is yet unclear. It affects various OS's (RHEL6 clones) of different versions and various boxes with or without control panels. 3.) In the absence of more solid information a lot of band-aids are currently being suggested. Like firewalling SSH and only allowing access from certain IP address ranges, or switching entirely to key based SSH logins. Which are good and sensible precautions to begin with. However, it doesn't fix the underlying problems, of which at least one is yet unknown. After posting the conclusions first, here is how they were obtained: a.) The topic of this thread and the one in WebHostingTalk (WHT) is a bit misleading. Yes, there is a recently discovered kernel vulnerability: CVE-2013-0871 https://bugzilla.redhat.com/show_bug.cgi?id=911937 https://access.redhat.com/security/cve/CVE-2013-0871 Race condition in the ptrace functionality in the Linux kernel before 3.7.5 allows local users to gain privileges via a PTRACE_SETREGS ptrace system call in a crafted application, as demonstrated by ptrace_death. This bug affects all kernels shipped with Red Hat Enterprise Linux 5, 6, Fedora Core and clones such as CentOS, Scientific Linux or Cloud Linux. The scope of the vulnerability is local. Means: An unprivileged user that already has local access to the box can gain root access through a complicated ptrace system call. Multi CPU boxes and timing issues probably make this a hell of a lot more complicated to exploit. b.) The WHT forums are a mess and always have been. The discussion about this topic (see http://www.webhostingtalk.com/showpost.php?p=8562898) is well beyond +50 pages now. There are some useful contributions to it, but the majority of the posts are useless repetitions or nitpickings about irrelevant details or wrong approaches. - If a server is hacked and the attacker gained "root" access, there is only one remedy: OS restore, fully patch, don't trust any data on the old box and start fresh. - A lot of the discussion on WHT is about how to "clean" a compromised box again, suggestion deletion of a library and fixing a symlink. Which is applying a band aid to an owned box which will get rooted again via the same approach vector. - What the attack mentioned in WHT does is this: http://www.webhostingtalk.com/showpost.php?p=8564042&postcount=329 Boxes with control panels that have been affected: cPanel, DirectAdmin, and Plesk, but I personally know of at least one BlueOnyx box that was most likely affected as well. I don't have access to that box anymore to verify this. The attacker can login via SSH to the box in a fashion that it will temporarily disable logging via syslog, __syslog_chk, audit_log_user_message, and audit_log_acct_message. The write() hook will also prevent logging via stderr. However, if the log level verbosity is raised to the maximum, then it'll show these logins as well. The common denominator is that most of the compromised boxes have been used for sending SPAM. When this happens, lots of SSH logins (which won't show up in the logs) are made and via remote command execution via SSH the emails are sent. While this attack is ongoing (i.e.: While the SSH connections happen) the process list and "lsof" might shed some light on this. However: In order to get to this point the attacker would already need "root" permission to substitute a library (/lib64/libkeyutils.so.1.9 or /lib/libkeyutils.so.1.9) that messes with SSH. Just the presence of this library doesn't spell doom, as there is a legitimate RPM that - in rare cases - might be installed and which brings a "good" version of that file aboard. A good reading is this blog page by John Williams, who sourced the crux of the info from WHT and supplemented it with his own findings: http://blog.solidshellsecurity.com/2013/02/18/0day-linuxcentos-sshd-spam-exploit-libkeyutils-so-1-9/ c.) Entry vector for this attack: As of now it is not 100% clear how the attacker(s) got access first. Did they first get unprivileged local access and then used the Kernel vulnerability CVE-2013-0871 to gain root access? Or is there another locally exploitable vulnerability that allows privilege escalation? At this time there is a lot of speculation going on about what the entry vector could be. This could be PHP related (which wouldn't surprise anyone). It could be a vulnerable daemon or (can't be ruled out entirely) even a problem with OpenSSH itself. While this is darn unlikely, it can't be ruled out entirely yet. There is even speculation about compromised credentials via the admin's PC. Which doesn't sound too far fetched if you figure in all the *very* serious Adobe and Java related vulnerabilities of the recent weeks and months. The bottom line is this: ======================== 1.) Current Linux kernel on RHEL 5 and 6 clones are vulnerable to a privilege escalation performed by a local user (or process). Updated kernels should hit the official vendor repositories soon. ETA unknown, though. 2.) SSHd Spam Exploit (libkeyutils.so.1.9): Aside from that there is currently an ongoing attack which started sometime in December 2012 and which seems to primarily target boxes with control panels. This attack modifies OpenSSH's behavior via a manually supplemented libkeyutils.so.1.9 library, which allows the attacker to login via SSH and to perform commands on the trojaned servers. It appears that this trojan also sends your "root" password to a certain collector box and that hacked servers are mostly used to send SPAM. The entry vector how this infection is/was carried out is yet unclear. It affects various OS's (RHEL6 clones) of different versions and various boxes with or without control panels. 3.) In the absence of more solid information a lot of band-aids are currently being suggested. Like firewalling SSH and only allowing access from certain IP address ranges, or switching entirely to key based SSH logins. Which are good and sensible precautions to begin with. However, it doesn't fix the underlying problems, of which at least one is yet unknown. --- Lastly: I'll watch WHT and other sources for more information about this and will post updates here if there is a major breakthrough. If you suspect that your BlueOnyx server is sending SPAMs and has either /lib64/libkeyutils.so.1.9 or /lib/libkeyutils.so.1.9 present, then please contact me offlist (!) and allow me to take a look at the box. -- With best regards Michael Stauber _______________________________________________ Blueonyx mailing list Blueonyx@mail.blueonyx.it http://mail.blueonyx.it/mailman/listinfo/blueonyx
