Interestingly, there was another failure mode not outlined in that summary. Mostly because it was "by design" and can only be considered a failure mode for users of Amazon's cloud, but not a failure of the cloud itself.
What we (Zeebox) noticed was that machines in other regions were going down as well. Most notably in us-west, but even some in eu-west. The machines in question were "spot" instances, where you bid a price and the real-time value of an instance is based on demand. If the value is below your bid, then you have a machine. When it goes over your bid, you lose it. It's an ideal model in may circumstances. I'll leave you to decide whether it worked exactly as it should in this instance, or if it can be classed as another level of the cascading failures :) As us-east went down, people turned to spot instances to make up for the lost capacity. In turn, this drove up the price, and anyone who had a spot instance happily doing its thing found themselves outbid and machine-less. And yes, it happened to us; though I'll add that we don't use spot instances for anything which would affect the user experience! They're better suited for continuous load testing and other similar tasks where a vanishing machine isn't too painful. On 5 July 2012 19:28, Carl of the Posse <[email protected]> wrote: > Amazon posted a nice summary of what went wrong with their systems: >> >> > http://aws.amazon.com/message/67457/ > > Problems with backup power, and then most importantly, problems with load > balancing control were what made the zone outage hard to work around. > > We (Netflix) might post a blog explaining how that affected us, the > internal issues that resulted and what we are doing about it. I'll reply to > this group if we do. > > > -- You received this message because you are subscribed to the Google Groups "Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
