"Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required."
"Anybody can ask a question. Anybody can answer. The best answers are voted up and rise to the top" http://serverfault.com/questions/769357/recovering-from-a-rm-rf Recovering from a rm -rf I run a small hosting provider with more or less 1535 customers and I use Ansible to automate some operations to be run on all servers. Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line. All servers got deleted and the offsite backups too because the remote storage was mounted just before by the same script (that is a backup maintenance script). How I can recover from a rm -rf / now in a timely manner? -- If you really don't have any backups I am sorry to say but you just nuked your entire company. – André Borie Apr 10 at 22:42 I feel sorry to say that your company is now essentially dead. You might have an extremely slim chance to recover from this if you turn off everything right now and hand your disks over to a reputable data recovery company. This will be extremely expensive and still extremely unlikely to really rescue you, and it will take a lot of time. – Sven♦ Apr 10 at 23:07 Backups need to be offsite, offline, and incremental. That you could delete them from your main server means they weren't what I would call backups. – Tim Apr 10 at 23:09 You're going out of business. You don't need technical advice, you need to call your lawyer. – Michael Hampton♦ Apr 10 at 23:10 @MarcoMarsala well at least you're famous now http://www.independent.co.uk/life-style/gadgets-and-tech/news/man-accidentally-deletes-his-entire-company-with-one-line-of-bad-code-a6984256.html (And at the above website:) "Man accidentally 'deletes his entire company' with one line of bad code" By Andrew Griffin, 19 hours ago ‘I feel sorry to say that your company is now essentially dead,’ one person on a coding forum advised Marco Marsala A man appears to have deleted his entire company with one mistaken piece of code. By accidentally telling his computer to delete everything in his servers, hosting provider Marco Marsala has seemingly removed all trace of his company and the websites that he looks after for his customers. Mr Marsala wrote on a forum for server experts called Server Fault that he was now stuck after having accidentally run destructive code on his own computers. But far from advising them how to fix it, most experts informed him that he had just accidentally deleted the data of his company and its clients, and in so doing had probably destroyed his entire company with just one line of code. The problem command was "rm -rf": a basic piece of code that will delete everything it is told to. The “rm” tells the computer to remove; the r deletes everything within a given directory; and the f stands for “force”, telling the computer to ignore the usual warnings that come when deleting files. Together, the code deleted everything on the computer, including Mr Masarla’s customers' websites, he wrote. Mr Masarla runs a web hosting company, which looks after the servers and internet connections on which the files for websites are stored. That piece of code is so famously destructive that it has become a joke within some computing circles. Normally, that code would wipe out all of the specific parts of the computer that it was pointed at. But because of an error in the way it was written, the code didn’t actually specify anywhere – and so removed everything on the computer. “I run a small hosting provider with more or less 1535 customers and I use Ansible to automate some operations to be run on all servers,” wrote Marco Marsala. “Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line.” Mr Marsala confirmed that the code had even deleted all of the backups that he had taken in case of catastrophe. Because the drives that were backing up the computers were mounted to it, the computer managed to wipe all of those, too. “All servers got deleted and the offsite backups too because the remote storage was mounted just before by the same script (that is a backup maintenance script).” Most users agreed that it was unlikely that Mr Marsala would be able to recover any of the data. And as a result his company was almost certainly not going to recover, either. “I feel sorry to say that your company is now essentially dead,” wrote a user called Sven. “You might have an extremely slim chance to recover from this if you turn off everything right now and hand your disks over to a reputable data recovery company. “This will be extremely expensive and still extremely unlikely to really rescue you, and it will take a lot of time.” Others agreed that perhaps Mr Marsala was on the wrong forum. “You're going out of business,” wrote Michael Hampton. “You don't need technical advice, you need to call your lawyer.” Many of the responses to Mr Marsala’s problem weren’t especially helpful – pointing out that he could have taken steps to stop it happening before it did. “Well, you should have been thinking about how to protect your customers' data before nuking them,” wrote one person calling himself Massimo. “I won't even begin enumerating how many errors are simultaneously required in order to be able to completely erase all your servers and all your backups in a single strike. “This is not bad luck: it's astonishingly bad design reinforced by complete carelessness.” Mr Marsala’s problem is far from the first time that someone has accidentally destroyed their own system by missing a mistake. Indeed, responses to his post mostly referenced a similar thread posted two years ago, with the headline “Monday morning mistake”. That error saw someone accidentally lose access to their entire server, after they didn’t notice a stray space in the code. == Cheers, Stephen _______________________________________________ Link mailing list [email protected] http://mailman.anu.edu.au/mailman/listinfo/link
