On Thu, Jul 10, 2014 at 10:09 AM, Siebrand Mazeland <siebr...@kitano.nl> wrote:
> This is an email to shell account holders on translatewiki.net and to > wikitech-l, so that you are informed. > > Today at 08:10 UTC Niklas noticed that the translatewiki.net server had > been compromised. We saw some suspicious files in /tmp and a few processes > that didn't belong: > > elastic+ 22862 0.0 0.0 2684 2388 ? S 04:53 0:00 > /tmp/freeBSD /tmp/freeBSD 1 > elastic+ 31575 0.0 0.0 2684 2388 ? S 06:38 0:00 > /tmp/freeBSD /tmp/freeBSD 1 > elastic+ 31580 16.7 0.0 90816 724 ? Ssl 06:38 16:26 > [.Linux_time_y_2] > > We gathered data and looked at our recent traffic statistics. We drew the > following conclusions: > > - Only the Elasticsearch account had been compromised. The intruder did not > gain access to other accounts. > - The attack could be made because the Elasticsearch process was bound to > all interfaces, instead of only the localhost interface, and dynamic > scripting was enabled, because it is required by CirrusSearch > (CVE-2014-3120). > - A virtual machine was started, and given the traffic that was generated > (about 1TB in the past 4 days), we think this was a DDoS drone. The process > reported to an IP address in China. > - A server reinstall is the right thing to do (better safe than sorry). > > The compromised server was taken off-line around 10:00 UTC today. > > Actions taken: > - Bind Elasticsearch only to localhost from now on: > https://gerrit.wikimedia.org/r/#/c/145262/ > - Reinstall the server > > Actions to be taken: > - Configure a firewall to only allow expected traffic to enter and exit the > translatewiki.net server so that something like the added virtual machine > could not have communicated to the outside world. > - As a precaution, shell account holders should change any secret that they > have used on the translatewiki.net server in the past 7 days. > > Did this server have access to private ssh keys that are used to push/merge code for upstream repos? If so, will they be rotated as well? - Ryan _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
