Here's an update for everyone who has been affected by the hosting servers being down.
The servers we use to host the loco teams are not in the Canonical data center. Instead, they're hosted in a server farm in the US where we rent space. For the most part, these machines are managed by the various teams who use them. Unfortunately, we were made aware yesterday (Monday, Aug 6th) that one of the machines had been compromised and was being used to try and hack other machines. We took a moment to assess the risk and evaluate the situation, then quickly took the machine down. The machine was known as 'ganges' and was the host to dc.ubuntu-us.org and a few other sites, most of which are not very active. We analyzed the other machines we rent in that data center and found a few to be acting suspiciously. Because of the late hour, we chose to shut down four other servers so that we can safely inspect them. These four machines are known as 'hudson', which is used by the doc team and a couple of the more active loco team websites (Sweden and Finland), also 'zambezi', which hosted the bulk of the smaller loco teams, 'parana', used by the software freedom day team and 'tiber', used by motu-revu. As part of the recovery process, we'd like to migrate the services running on these machines from the rented servers to machines in the Canonical Data Centre. This will give several benefits: o Immediate upgrade to a security supported release of Ubuntu o Greatly improved hardware o Full time sysadmin support from Canonical It will however come with one restriction which the external servers don't have and that is that full root access won't be available. FWIw, The Italian team already migrated to a server in the Canonical DC a while ago. Unfortunately the recovery process will take some time. We've asked the hosting company to boot the servers up into Live CDs already. Once they've done that we'll recover the data and content from the existing machines and re-setup the services on new machines. We'll keep you updated with progres reports and an ETA as soon as we have one. If you have a very recent backup of your site, we can probably get your site up quicker. Please notify me. Of greater concern, please consider carefully if you are a user of 'ganges' or one of the suspected compromised servers. Any sensitive files (e.g. gpg or ssh keys) or data (e.g. typed passwords, forwarded SSH keys) stored on or that passed through those servers recently should be considered compromised and revoked & new ones generated. You may want to check your machine and any machine you logged into from the (potentially) compromised servers. Here is a webpage that contains more details, a bit on the paranoid side, about what steps you should consider taking. <http://www.wiggy.net/debian/developer-securing/> If you have any concerns or questions, please refer to the above link. If that does not answer your question, please contact me. It's too early to be sure, but at the moment we're not expecting any data loss. -- Matthew Nuzum newz2000 on freenode -- loco-contacts mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/loco-contacts
